This repository has been archived by the owner on Oct 9, 2023. It is now read-only.
Handle WorkflowClosure from storage #459
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8 tasks
ckiosidis
force-pushed
the
add-data-reference-fields-to-crd
branch
2 times, most recently
from
3167be8 to
36b39cd
Compare
ckiosidis
force-pushed
the
add-data-reference-fields-to-crd
branch
8 times, most recently
from
7c81a43 to
840c098
Compare
ckiosidis
commented
Jul 25, 2022
ckiosidis
commented
Jul 25, 2022
ckiosidis
commented
Jul 25, 2022
ckiosidis
changed the title
lupasarin
reviewed
Jul 26, 2022
| assert.Equal(t, 1, len(r.Finalizers)) | ||
| assert.False(t, HasCompletedLabel(r)) | ||
| assert.Equal(t, uint32(0), r.Status.FailedAttempts) | ||
| assert.Nil(t, r.WorkflowSpec) |
There was a problem hiding this comment.
Shouldn't it return the one with ID: "static-id"? instead of nil
ckiosidis
force-pushed
the
add-data-reference-fields-to-crd
branch
5 times, most recently
from
9525a8a to
f877e36
Compare
ckiosidis
changed the title
hamersaw
suggested changes
Aug 8, 2022
ckiosidis
force-pushed
the
add-data-reference-fields-to-crd
branch
2 times, most recently
from
f7a0c13 to
1b4c6cf
Compare
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
and clear fields before updating the status Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Babis Kiosidis <[email protected]>
Co-authored-by: Ketan Umare <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
ckiosidis
force-pushed
the
add-data-reference-fields-to-crd
branch
from
33350b5 to
477c445
Compare
eapolinario
pushed a commit
to eapolinario/flytepropeller
that referenced
this pull request
Aug 9, 2023
* add data reference fields Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * combine crd parts into one object/location Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * create and pass static obj storage to propeller handler Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * read static blob at the beginning and clear fields before updating the status Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * formatting Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * pass tests Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * remove terminated wfs' blob obj from cache Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * no return on remove blob method Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * test happy offloaded spec scenario Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * set static fields on every streak Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * formatting Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * added crdOffloadStore interface Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * made crdoffloadstore configurable Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * add metrics/formatting Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * load static workflow data outside streak loop Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * cleaned up metric reporting Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * renamed inmemory to active Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * added lruCRDOffloadStore unit tests Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * added activeCRDOffloadStore unit tests Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * added unit test for offloading crd error on handle Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * handle offloaded WorkflowClosure instead of parts of the crd Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * cache wf crd fields instead of wf closure Signed-off-by: Babis Kiosidis <[email protected]> * Update pkg/controller/controller.go Co-authored-by: Ketan Umare <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * reading workflow closure directly from data store rather than cache Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * added prometheus metric Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * updated comments Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Co-authored-by: Babis Kiosidis <[email protected]> Co-authored-by: Daniel Rammer <[email protected]> Co-authored-by: Ketan Umare <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Extends the FlyteWorkflow crd to contain a location reference to the workflowClosure is stored as protobuf
Introduces a in memory cache of loading the static json fields and reusing them in the FlyteWorkflow CRD if the DataReference is present.
TL;DR
Please replace this text with a description of what this PR accomplishes.
Type
Are all requirements met?
Complete description
How did you fix the bug, make the feature etc. Link to any design docs etc
Tracking Issue
flyteorg/flyte#2705
Follow-up issue
NA
OR
https://github.com/flyteorg/flyte/issues/