Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address resolution #1567

Merged
merged 11 commits into from
May 17, 2023
Merged

Address resolution #1567

merged 11 commits into from
May 17, 2023

Conversation

wild-endeavor
Copy link
Contributor

@wild-endeavor wild-endeavor commented Mar 29, 2023

TL;DR

This implements in FlyteRemote a quick feature to help the fetching of inputs and outputs and decks from a flyte:// uri.

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

Other PRs:
flyteorg/flyteadmin#546
flyteorg/flyteidl#391

Tracking Issue

No issues, this is just a use-ability enhancement we found

Signed-off-by: Yee Hing Tong <[email protected]>
@codecov
Copy link

codecov bot commented Mar 29, 2023

Codecov Report

Merging #1567 (0b7ee45) into master (c65397d) will decrease coverage by 0.05%.
The diff coverage is 42.18%.

@@            Coverage Diff             @@
##           master    #1567      +/-   ##
==========================================
- Coverage   71.04%   71.00%   -0.05%     
==========================================
  Files         335      336       +1     
  Lines       30661    30713      +52     
  Branches     5558     5565       +7     
==========================================
+ Hits        21783    21807      +24     
- Misses       8336     8362      +26     
- Partials      542      544       +2     
Impacted Files Coverage Δ
flytekit/clients/friendly.py 4.89% <0.00%> (-0.15%) ⬇️
flytekit/clients/raw.py 7.37% <0.00%> (-0.13%) ⬇️
flytekit/tools/interactive.py 0.00% <0.00%> (ø)
flytekit/remote/remote.py 39.39% <32.25%> (-0.48%) ⬇️
flytekit/deck/deck.py 48.64% <50.00%> (+4.74%) ⬆️
tests/flytekit/unit/deck/test_deck.py 100.00% <100.00%> (ø)
tests/flytekit/unit/remote/test_remote.py 99.45% <100.00%> (+0.04%) ⬆️

…his is going away anyways

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
flytekit/remote/remote.py Outdated Show resolved Hide resolved
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
@wild-endeavor wild-endeavor changed the title [wip] address resolution Address resolution May 2, 2023
@wild-endeavor wild-endeavor marked this pull request as ready for review May 2, 2023 20:44
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
pingsutw
pingsutw previously approved these changes May 16, 2023
Signed-off-by: Yee Hing Tong <[email protected]>
@wild-endeavor wild-endeavor merged commit 01830e4 into master May 17, 2023
eapolinario pushed a commit that referenced this pull request Jul 10, 2023
Signed-off-by: Yee Hing Tong <[email protected]>
eapolinario added a commit that referenced this pull request Jul 12, 2023
* Multi arch imageSpec (#1630)

Multi arch imageSpec (#1630)

Signed-off-by: Eduardo Apolinario <[email protected]>

* Add executor_path and applications_path to spark config (#1634)

* Add executor_path and applications_path to spark config

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Add support for env vars to pyflyte run (#1617)

* Add support for env vars to pyflyte run

Signed-off-by: Kevin Su <[email protected]>

* bump idl

Signed-off-by: Kevin Su <[email protected]>

* update doc

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>

* Fetch task executions in dynamic  (#1636)

* fetch task executions in dynamic

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Added metrics command to pyflyte (#1513)

Signed-off-by: Daniel Rammer <[email protected]>

* Add http_proxy to client & Fix deviceflow (#1611)

* Add http_proxy to client & Fix deviceflow

RB=3890720

Signed-off-by: byhsu <[email protected]>

* nit

Signed-off-by: byhsu <[email protected]>

* lint!

Signed-off-by: byhsu <[email protected]>

---------

Signed-off-by: byhsu <[email protected]>
Co-authored-by: byhsu <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>

* Improve variable names (#1642)

Signed-off-by: byhsu <[email protected]>
Co-authored-by: byhsu <[email protected]>

* Address resolution (#1567)

Signed-off-by: Yee Hing Tong <[email protected]>

* pyflyte run supports pickle (#1646)

Signed-off-by: Kevin Su <[email protected]>

* Wait for the pod plugin instead of flytekit (#1647)

Signed-off-by: eduardo apolinario <[email protected]>
Co-authored-by: eduardo apolinario <[email protected]>

* Beautify deviceflow prompt (#1625)

* Beautify deviceflow prompt

Signed-off-by: byhsu <[email protected]>

* lint!

Signed-off-by: byhsu <[email protected]>

* lint

Signed-off-by: byhsu <[email protected]>

---------

Signed-off-by: byhsu <[email protected]>
Co-authored-by: byhsu <[email protected]>

* Improve flytekit register (#1643)

* Fix pyflyte register

Signed-off-by: byhsu <[email protected]>

* revert

Signed-off-by: byhsu <[email protected]>

* lint

Signed-off-by: byhsu <[email protected]>

---------

Signed-off-by: byhsu <[email protected]>
Co-authored-by: byhsu <[email protected]>

* Pass verify flag to all authenticators (#1641)

Signed-off-by: byhsu <[email protected]>

* Allow annotated FlyteFile as task input argument (#1632)

* fix: Allow annotated FlyteFile as task input argument

Using an annotated FlyteFile type as an input to a task was previously impossible due
to an exception being raised in `FlyteFilePathTransformer.to_python_value`.

This commit applies the fix previously used in `FlyteFilePathTransformer.to_literal`
to permit using annotated FlyteFiles as either inputs and outputs of a task.

Issue: #3424
Signed-off-by: Adrian Rumpold <[email protected]>

* refactor: Unified handling of annotated types in type engine

Issue: #3424
Signed-off-by: Adrian Rumpold <[email protected]>

* fix: Use py3.8-compatible types in type engine tests

Issue: #3424
Signed-off-by: Adrian Rumpold <[email protected]>

---------

Signed-off-by: Adrian Rumpold <[email protected]>

* Use logger instead of print statement in sqlalchemy plugin (#1651)

* use logging info instead of print

Signed-off-by: wirthual <[email protected]>

* isorted files

Signed-off-by: wirthual <[email protected]>

* import root logger from flytekit

Signed-off-by: wirthual <[email protected]>

---------

Signed-off-by: wirthual <[email protected]>

* Map over notebook task (#1650)

* map over notebook

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* tests

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* add a flag

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* fix tests

Signed-off-by: Kevin Su <[email protected]>

* nit

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

* Fix tests

Signed-off-by: Kevin Su <[email protected]>

* lint

Signed-off-by: Kevin Su <[email protected]>

---------

Signed-off-by: Kevin Su <[email protected]>

* Support single literals in tiny url (#1654)

Signed-off-by: Yee Hing Tong <[email protected]>

* Add support overriding image (#1652)

Signed-off-by: Kevin Su <[email protected]>

* Fix ability to pass None to task with Optional kwarg, add test (#1657)

Signed-off-by: Fabio Grätz <[email protected]>
Co-authored-by: Fabio Grätz <[email protected]>

* Regenerate plugins requirements

Signed-off-by: eduardo apolinario <[email protected]>

* Regenerate plugins requirements and linting

Signed-off-by: eduardo apolinario <[email protected]>

* Regenerate whylogs requirements

Signed-off-by: eduardo apolinario <[email protected]>

---------

Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]>
Signed-off-by: byhsu <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: eduardo apolinario <[email protected]>
Signed-off-by: Adrian Rumpold <[email protected]>
Signed-off-by: wirthual <[email protected]>
Signed-off-by: Fabio Grätz <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Co-authored-by: Dan Rammer <[email protected]>
Co-authored-by: ByronHsu <[email protected]>
Co-authored-by: byhsu <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>
Co-authored-by: eduardo apolinario <[email protected]>
Co-authored-by: Adrian Rumpold <[email protected]>
Co-authored-by: wirthual <[email protected]>
Co-authored-by: Fabio M. Graetz, Ph.D <[email protected]>
Co-authored-by: Fabio Grätz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants