Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ignore cache variables #4618

Merged
merged 7 commits into from
Dec 19, 2023
Merged

Conversation

hamersaw
Copy link
Contributor

Tracking issue

Iterating on PR - #4324

Why are the changes needed?

Refer to linked PR.

What changes were proposed in this pull request?

Updated to disable input vars in the HashLiteralMap function. This makes it compatible with the existing maptask implementation which uses the async catalog client.

How was this patch tested?

Refer to linked PR.

Setup process

Refer to linked PR.

Screenshots

Refer to linked PR.

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

#4324

Docs link

Refer to linked PR.

@dosubot dosubot bot added size:XXL This PR changes 1000+ lines, ignoring generated files. enhancement New feature or request labels Dec 18, 2023
Copy link

codecov bot commented Dec 18, 2023

Codecov Report

Attention: 11 lines in your changes are missing coverage. Please review.

Comparison is base (14915dc) 59.03% compared to head (6ac3e91) 59.04%.

Files Patch % Lines
flyteplugins/go/tasks/plugins/array/catalog.go 0.00% 6 Missing ⚠️
flytepropeller/pkg/controller/nodes/task/cache.go 0.00% 5 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##           master    #4618   +/-   ##
=======================================
  Coverage   59.03%   59.04%           
=======================================
  Files         622      622           
  Lines       52682    52685    +3     
=======================================
+ Hits        31103    31108    +5     
  Misses      19097    19097           
+ Partials     2482     2480    -2     
Flag Coverage Δ
unittests 59.04% <52.17%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@pvditt pvditt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

Would it make sense to automatically evict task output for previous cached results for a task given the cache_ignore_input_vars changes? Can see it both ways as this is a little different than updating the params of a task.
(outside the context of the above PR but thinking as part of cache deletion work)

@hamersaw hamersaw merged commit b562536 into master Dec 19, 2023
44 of 45 checks passed
@hamersaw hamersaw deleted the troychiu/cache_ignore_portion/iterate branch December 19, 2023 14:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants