Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core feature] [flytekit] Using remote.fetch_task causes network calls - potentially on module load #3176

Closed
2 tasks done
kumare3 opened this issue Dec 20, 2022 · 1 comment
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@kumare3
Copy link
Contributor

kumare3 commented Dec 20, 2022

Motivation: Why do you think this is important?

There are 2 ways today in which users can use remote tasks in flyte.

  1. @reference_task/workflow - docs
  2. remote.fetch_task / fetch_workflow

Reference tasks are neverloaded till registration time and are nicer to work with.
but, often times it is desirable to simply construct a workflow by loading remote tasks. this can be done so

remote = FlyteRemote(...)
t1 = remote.fetch_task(...)
...
@workflow
def foo():
   t1(...)

The problem with this approach is that this will cause the t1 to be loaded a module load time.

Goal: What should the final outcome look like, ideally?

it should be possible to lazy load tasks from remote.

t1 = remote.fetch_task_lazy(....)
...

this should automatically load the task when used in a workflow at registration or local execution time.

Describe alternatives you've considered

NA

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@kumare3 kumare3 added enhancement New feature or request untriaged This issues has not yet been looked at by the Maintainers and removed untriaged This issues has not yet been looked at by the Maintainers labels Dec 20, 2022
@kumare3 kumare3 self-assigned this Dec 20, 2022
@kumare3 kumare3 added this to the 1.3.0 milestone Dec 20, 2022
@cosmicBboy
Copy link
Contributor

hey @kumare3 can we close this?

@kumare3 kumare3 closed this as completed Dec 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants