Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TypeEngine] Schema version priority for union dataclass comparison #2959

Conversation

mao3267
Copy link
Contributor

@mao3267 mao3267 commented Nov 26, 2024

Tracking issue

flyteorg/flyte#5489

Why are the changes needed?

Since we plan to primarily support newer versions of JSON schemas in the future, we would like to update the JSON schema version priority in the literal metadata field.

What changes were proposed in this pull request?

  1. We are modifying the priority for generating JSON schemas. Initially, the schema from Marshmallow, which follows draft-7, was prioritized. Moving forward, we will prioritize the schema from Mashumaro, which adheres to the newer draft 2020-12 specification.
  2. Update related unit tests to make sure everything works as expected.
  3. Fix missing logic for dict while generating attribute list in generate_attribute_list_from_dataclass_json_mixin function.

How was this patch tested?

  1. Pass existing tests and modified tests

Setup process

git clone https://github.com/flytorg/flytekit.git
gh pr checkout 2952
pip install -e .

Screenshots

image
image

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

#2932
#2859

Docs link

None

Future-Outlier and others added 5 commits November 15, 2024 10:26
Signed-off-by: Future-Outlier <[email protected]>
…er-for-union-dataclass-comparison' into schema-version-priority-for-union-dataclass-comparison
…ema-version-priority-for-union-dataclass-comparison
mao3267 and others added 2 commits November 28, 2024 22:25
@Future-Outlier Future-Outlier changed the title [WIP] Schema version priority for union dataclass comparison [TypeEngine] Schema version priority for union dataclass comparison Nov 29, 2024
Copy link
Member

@Future-Outlier Future-Outlier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!

@Future-Outlier Future-Outlier enabled auto-merge (squash) November 29, 2024 03:19
Copy link

codecov bot commented Nov 29, 2024

Codecov Report

Attention: Patch coverage is 14.28571% with 12 lines in your changes missing coverage. Please review.

Project coverage is 51.35%. Comparing base (d5ea440) to head (2e080aa).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
flytekit/core/type_engine.py 14.28% 12 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #2959       +/-   ##
===========================================
- Coverage   92.54%   51.35%   -41.20%     
===========================================
  Files           7      200      +193     
  Lines         483    20921    +20438     
  Branches        0     2693     +2693     
===========================================
+ Hits          447    10743    +10296     
- Misses         36     9579     +9543     
- Partials        0      599      +599     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Future-Outlier Future-Outlier merged commit 3cfe6cc into flyteorg:master Nov 29, 2024
29 of 31 checks passed
arbaobao pushed a commit to arbaobao/flytekit that referenced this pull request Dec 4, 2024
…lyteorg#2959)

* Union Dataclass Ambiguos Error

Signed-off-by: Future-Outlier <[email protected]>

* fix: change schema to 2020-12 from mashumaro

Signed-off-by: mao3267 <[email protected]>

* fix: handle dict type while generating attr from dataclass json mixin

Signed-off-by: mao3267 <[email protected]>

* fix: update comments and use title to check nested dataclass

Signed-off-by: mao3267 <[email protected]>

* LGTM

Signed-off-by: Future-Outlier <[email protected]>

---------

Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: mao3267 <[email protected]>
Co-authored-by: Future-Outlier <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants