-
Notifications
You must be signed in to change notification settings - Fork 301
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Flytekitplugin pandera update: use entrypoint and structured dataset #2821
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
cosmicBboy
requested review from
wild-endeavor,
kumare3,
eapolinario,
pingsutw,
samhita-alla,
thomasjpfan and
Future-Outlier
as code owners
October 16, 2024 13:23
cosmicBboy
changed the title
Flytekitplugin pandera fix
Flytekitplugin pandera update: use entrypoint and structured dataset
Oct 16, 2024
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2821 +/- ##
==========================================
+ Coverage 76.82% 77.22% +0.40%
==========================================
Files 196 246 +50
Lines 20301 22269 +1968
Branches 2610 2609 -1
==========================================
+ Hits 15596 17198 +1602
- Misses 4004 4317 +313
- Partials 701 754 +53 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
eapolinario
reviewed
Oct 16, 2024
plugins/flytekit-pandera/flytekitplugins/pandera/pandas_transformer.py
Outdated
Show resolved
Hide resolved
plugins/flytekit-pandera/flytekitplugins/pandera/pandas_transformer.py
Outdated
Show resolved
Hide resolved
Signed-off-by: Niels Bantilan <[email protected]>
Signed-off-by: Niels Bantilan <[email protected]>
eapolinario
approved these changes
Oct 16, 2024
pingsutw
approved these changes
Oct 16, 2024
kumare3
pushed a commit
that referenced
this pull request
Nov 8, 2024
…2821) * fix pandera plugin error Signed-off-by: Niels Bantilan <[email protected]> * add validation config Signed-off-by: Niels Bantilan <[email protected]> * use structured dataset transformer Signed-off-by: Niels Bantilan <[email protected]> * refactor get_literal_type Signed-off-by: Niels Bantilan <[email protected]> * cleanup Signed-off-by: Niels Bantilan <[email protected]> * add unit tests Signed-off-by: Niels Bantilan <[email protected]> * remove unused import Signed-off-by: Niels Bantilan <[email protected]> * fix lint Signed-off-by: Niels Bantilan <[email protected]> * fix lint Signed-off-by: Niels Bantilan <[email protected]> * use logger, remove unneeded typing for <py38 Signed-off-by: Niels Bantilan <[email protected]> * fix lint Signed-off-by: Niels Bantilan <[email protected]> --------- Signed-off-by: Niels Bantilan <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tracking issue
flyteorg/flyte#5851
Why are the changes needed?
The pandera flytekit plugin is out-dated and needs to be maintained.
What changes were proposed in this pull request?
entrypoint
config insetup.py
for auto-registration of the type transformerschema.py -> pandas_transformer.py
PandasReportRenderer
ValidationConfig
withon_error
option. This is so thaton_error="warn"
can create a Flyte deck that's actually rendered in the UI. Ideallyon_error="raise"
will also render a deck, but right now there's a limitation where decks aren't rendered with failed tasks.How was this patch tested?
Update the unit tests, run with
pytest test
in the plugin directory.Check all the applicable boxes