Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve literal type string representation handling #5932

Merged
merged 16 commits into from
Nov 1, 2024
Merged

Conversation

pingsutw
Copy link
Member

Tracking issue

NA

Why are the changes needed?

Printing the JSON schema in the error message is not helpful, and sometimes the error message becomes too large when using a large dataclass

What changes were proposed in this pull request?

Add a LiteralTypeToStr function to convert LiteralType to string. It will remove the metadata from Literal Type if the type is Simple.Struct.

How was this patch tested?

from dataclasses import dataclass

from flytekit import task, workflow, ImageSpec
from flytekit.configuration import Config
from flytekit.remote import FlyteRemote

remote = FlyteRemote(Config.auto(), default_project="flytesnacks", default_domain="development", interactive_mode_enabled=True)
new_flytekit = "git+https://github.com/flyteorg/flytekit.git@8017ae825b7c4cc3913971fab5308b172370d335"
img = ImageSpec(registry="pingsutw", packages=[new_flytekit], apt_packages=["git"])


@dataclass
class SecondDatclass:
    name: str
    age: int


@dataclass
class MyDatclass:
    id: int
    name: str
    age: int
    second_dataclass: SecondDatclass


@task(container_image=img)
def hello(dc: MyDatclass) -> str:
    return f"Hello {dc.name}"


@workflow
def wf(dc: MyDatclass = MyDatclass(id=1, name="foo", age=19, second_dataclass=SecondDatclass(name="foo", age=18))) -> str:
    return hello(dc=dc.name)

Setup process

pyflyte register --version v1 flyte-example/improve_error/dataclass_mismatch.py

Screenshots

Before:
Screenshot 2024-10-28 at 12 31 28 PM

After:
Screenshot 2024-10-28 at 12 32 02 PM

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Copy link

codecov bot commented Oct 28, 2024

Codecov Report

Attention: Patch coverage is 77.77778% with 4 lines in your changes missing coverage. Please review.

Project coverage is 36.82%. Comparing base (13b3d82) to head (e5c2af1).
Report is 12 commits behind head on master.

Files with missing lines Patch % Lines
...epropeller/pkg/compiler/transformers/k8s/inputs.go 0.00% 1 Missing ⚠️
flytepropeller/pkg/compiler/validators/bindings.go 66.66% 1 Missing ⚠️
...lytepropeller/pkg/compiler/validators/condition.go 0.00% 1 Missing ⚠️
flytepropeller/pkg/compiler/validators/vars.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5932      +/-   ##
==========================================
- Coverage   36.84%   36.82%   -0.03%     
==========================================
  Files        1309     1310       +1     
  Lines      130967   130979      +12     
==========================================
- Hits        48252    48229      -23     
- Misses      78531    78572      +41     
+ Partials     4184     4178       -6     
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin 54.14% <ø> (+0.02%) ⬆️
unittests-flytecopilot 11.73% <ø> (ø)
unittests-flytectl 62.40% <ø> (ø)
unittests-flyteidl 6.92% <ø> (ø)
unittests-flyteplugins 53.64% <ø> (ø)
unittests-flytepropeller 42.88% <77.77%> (-0.12%) ⬇️
unittests-flytestdlib 55.41% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Copy link

github-actions bot commented Oct 28, 2024

Dependency Review

The following issues were found:

  • ❌ 3 vulnerable package(s)
  • ✅ 0 package(s) with incompatible licenses
  • ✅ 0 package(s) with invalid SPDX license definitions
  • ⚠️ 32 package(s) with unknown licenses.

View full job summary

Signed-off-by: Kevin Su <[email protected]>
This reverts commit dd43659.
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
@pingsutw
Copy link
Member Author

cc @pmahindrakar-oss

Signed-off-by: Kevin Su <[email protected]>
Copy link
Member

@Future-Outlier Future-Outlier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing, and it's backward compatible (really happy about this).
Tell me when you want to merge it (need to fix lint error)

Signed-off-by: Kevin Su <[email protected]>
Copy link
Member

@Future-Outlier Future-Outlier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very nice PR, thank you

@pingsutw pingsutw merged commit a2988ba into master Nov 1, 2024
50 of 51 checks passed
@pingsutw pingsutw deleted the prtty-print branch November 1, 2024 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants