Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Resolve Attr Path for List or Dict of Promise #2828

Merged
merged 4 commits into from
Oct 18, 2024

Conversation

Mecoli1219
Copy link
Contributor

@Mecoli1219 Mecoli1219 commented Oct 17, 2024

Tracking issue

flyteorg/flyte#5856
flyteorg/flyte#5593

Why are the changes needed?

We should support this:

@dataclass
class DC:
    a: int

@dataclass
class DC2:
    a: DC
    b: int

@task(container_image=image)
def task_1(in1: int) -> typing.List[int]:
    return list(range(max(2, in1)))

@task(container_image=image)
def task_2(in1: int, in2: typing.Dict[str, typing.List[int]]) -> int:
    for k, v in in2.items():
        in1 += sum(v)
    return in1

@task(container_image=image)
def task_3() -> DC2:
    return DC2(a=DC(a=1), b=2)

@workflow
def my_wf(in1: int, in2: int) -> int:
    a = task_1(in1=in1)
    a1 = a[0]
    dc = task_3()
    b = task_2(in1=5, in2={"a": [a1, dc["b"]], "b": [dc.a["a"], in2]})
    return b

What changes were proposed in this pull request?

If the input is the list, call resolve_attr_path_in_promise for each element of that.

How was this patch tested?

  1. Run the code above locally & remotely.
  2. A new unit test is added

Setup process

Screenshots

image

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Signed-off-by: Mecoli1219 <[email protected]>
@Mecoli1219
Copy link
Contributor Author

cc @Future-Outlier

@Future-Outlier
Copy link
Member

cc @Future-Outlier

Yes you are amazing

Copy link

codecov bot commented Oct 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.87%. Comparing base (37e4fad) to head (38ba736).
Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2828      +/-   ##
==========================================
- Coverage   80.57%   78.87%   -1.71%     
==========================================
  Files         260      196      -64     
  Lines       23778    20344    -3434     
  Branches     2618     2622       +4     
==========================================
- Hits        19160    16047    -3113     
+ Misses       3922     3575     -347     
- Partials      696      722      +26     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

flytekit/core/promise.py Outdated Show resolved Hide resolved
tests/flytekit/unit/core/test_promise.py Show resolved Hide resolved
Signed-off-by: Mecoli1219 <[email protected]>
pingsutw
pingsutw previously approved these changes Oct 17, 2024
flytekit/core/promise.py Outdated Show resolved Hide resolved
Signed-off-by: Mecoli1219 <[email protected]>
Signed-off-by: Mecoli1219 <[email protected]>
@Mecoli1219 Mecoli1219 changed the title Enable Resolve Attr Path for List of Promise Enable Resolve Attr Path for List or Dict of Promise Oct 17, 2024
@Mecoli1219
Copy link
Contributor Author

@pingsutw I updated the function to remove some redundant code and also support the Dictionary. Could you review it for me? Thx!

@Future-Outlier
Copy link
Member

testing this PR, let me merge it.

@Future-Outlier
Copy link
Member

It's great.
image

Do you want to also add a test for dict[str, type]?
@Mecoli1219

@Future-Outlier Future-Outlier merged commit 816015d into flyteorg:master Oct 18, 2024
104 checks passed
@Mecoli1219
Copy link
Contributor Author

Do you want to also add a test for dict[str, type]?

@Future-Outlier I updated it in the description

@Future-Outlier Future-Outlier self-assigned this Oct 28, 2024
kumare3 pushed a commit that referenced this pull request Nov 8, 2024
* enable attr path to list

Signed-off-by: Mecoli1219 <[email protected]>

* enable nested list

Signed-off-by: Mecoli1219 <[email protected]>

* nit

Signed-off-by: Mecoli1219 <[email protected]>

* Enable dict support

Signed-off-by: Mecoli1219 <[email protected]>

---------

Signed-off-by: Mecoli1219 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants