Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gate node in dynamic task #729

Merged
merged 10 commits into from
Mar 30, 2023
Merged

gate node in dynamic task #729

merged 10 commits into from
Mar 30, 2023

Conversation

james-union
Copy link
Contributor

@james-union james-union commented Mar 22, 2023

flyteorg/flyte#3354

How to reproduce

flytesnacks/development/human_in_the_loop.wf_dyn

Launch the above workflow and it should show Paused for gate nodes.

How to fix

Dynamic nodes are included in subWorkflows under compiledWorkflowClosure instead of primary. We only considered priamry to check the compiled nodes and that's why the ui couldn't understand the gate nodes.

Another issue was that compiled nodes are not only based on execution node id. We should also compare with the metadata.specNodeId

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

How did you fix the bug, make the feature etc. Link to any design docs etc

Tracking Issue

Remove the 'fixes' keyword if there will be multiple PRs to fix the linked issue

fixes flyteorg/flyte#3354

@james-union james-union self-assigned this Mar 22, 2023
@codecov
Copy link

codecov bot commented Mar 22, 2023

Codecov Report

Merging #729 (f3339c9) into master (6ebd69f) will increase coverage by 0.13%.
The diff coverage is 80.48%.

@@            Coverage Diff             @@
##           master     #729      +/-   ##
==========================================
+ Coverage   66.61%   66.74%   +0.13%     
==========================================
  Files         464      464              
  Lines       11370    11388      +18     
  Branches     2094     2102       +8     
==========================================
+ Hits         7574     7601      +27     
+ Misses       3796     3787       -9     
Impacted Files Coverage Δ
...ponents/Executions/Tables/nodeExecutionColumns.tsx 31.81% <0.00%> (-0.49%) ⬇️
...ents/flytegraph/ReactFlow/customNodeComponents.tsx 19.18% <25.00%> (+0.47%) ⬆️
...s/flytegraph/ReactFlow/ReactFlowGraphComponent.tsx 52.00% <50.00%> (+0.48%) ⬆️
...ponents/Executions/Tables/NodeExecutionActions.tsx 89.70% <80.00%> (-1.34%) ⬇️
packages/console/src/components/hooks/utils.ts 94.44% <83.33%> (-5.56%) ⬇️
...tions/ExecutionDetails/ExecutionDetailsActions.tsx 82.60% <100.00%> (+0.79%) ⬆️
...cutionDetails/NodeExecutionDetailsPanelContent.tsx 81.57% <100.00%> (+0.19%) ⬆️
...mponents/Executions/Tables/NodeExecutionsTable.tsx 91.80% <100.00%> (+0.13%) ⬆️
...ons/contextProvider/NodeExecutionDetails/index.tsx 72.85% <100.00%> (+0.39%) ⬆️
...ackages/console/src/components/Executions/utils.ts 68.75% <100.00%> (-0.28%) ⬇️
... and 1 more

... and 1 file with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@ursucarina
Copy link
Contributor

ursucarina commented Mar 22, 2023

@james-union 2 things:

  • clicking Resume doesn't do anything, because 'compiledNode' is undefined. Is this expected?
  • can you please up the package version
  • please remember to sign off on all your commits git commit --amend --signoff

@james-union james-union force-pushed the james/gate-node-in-dynamic branch from d575644 to 279ca10 Compare March 23, 2023 12:22
@jsonporter
Copy link
Contributor

jsonporter commented Mar 24, 2023

This PR fixes identifying gate nodes however I don't think we can merge this without the other fix for [BUG] Flyteconsole: status of gate nodes does not update without refresh #3524 and #3443 because the experience is still broken.

@@ -23,9 +23,6 @@ export function makeWorkflowQuery(

return workflow;
},
// `Workflow` objects (individual versions) are immutable and safe to
// cache indefinitely once retrieved in full
staleTime: Infinity,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this does work in fixing the issue where the graph errantly shows the nested contents (when it should instead show the grey-subworkflow button; see image) however I don't think this is right place to make the fix because these values will be immutable meaning we should not be invalidating the request cache. That said - I agree that this is likely a caching issue, just not here

correct:
Screenshot 2023-03-28 at 10 10 01 AM

@jsonporter jsonporter merged commit ca1b5b8 into master Mar 30, 2023
@jsonporter jsonporter deleted the james/gate-node-in-dynamic branch March 30, 2023 17:56
4nalog pushed a commit that referenced this pull request Mar 30, 2023
* fix: gate node in dynamic

Signed-off-by: James <[email protected]>

* fix: upgraded version

Signed-off-by: James <[email protected]>

* fix: compiledNode in PausedTasksComponent and ExecutionDetailsActions

Signed-off-by: James <[email protected]>

* fix: upgrade version

Signed-off-by: James <[email protected]>

* chore: lockfile

Signed-off-by: Carina Ursu <[email protected]>

* fix: remove caching for workflow closure

Signed-off-by: James <[email protected]>

* fix: break link between cache and context state

Signed-off-by: James <[email protected]>

---------

Signed-off-by: James <[email protected]>
Signed-off-by: Carina Ursu <[email protected]>
Co-authored-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>
jsonporter added a commit that referenced this pull request Mar 30, 2023
* fix: type error on invalid nested json input

Signed-off-by: 4nalog <[email protected]>

* LaunchForm RJSF Form issue (#692)

* fix: launchform

Signed-off-by: James <[email protected]>

* fix: stuck issue

Signed-off-by: James <[email protected]>

* fix: on form change

Signed-off-by: James <[email protected]>

* fix: merge master into branch

Signed-off-by: James <[email protected]>

---------

Signed-off-by: James <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* chore: bump minor version

Signed-off-by: 4nalog <[email protected]>

* fix: package version to 1.4.2 (#700)

Signed-off-by: Jason Porter <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: project settings dashbboard tests (#701)

* fix: project settings dashbboard tests

Signed-off-by: 4nalog <[email protected]>

* chore: remove .only

Signed-off-by: 4nalog <[email protected]>

* refactor: remove empty waitFor

Signed-off-by: 4nalog <[email protected]>

---------

Signed-off-by: 4nalog <[email protected]>

* fix: upgrading node version to 18 (#703)

Upgrading node version to 18

Signed-off-by: Jason Porter <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: revert node version (#704)

* fix: revert node version

Signed-off-by: Carina Ursu <[email protected]>

* Revert "fix: upgrading node version to 18 (#703)"

This reverts commit e5f9a8b.

Signed-off-by: Carina Ursu <[email protected]>

---------

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: deployment optimization (#706)

* fix: deployment optimization

Signed-off-by: Carina Ursu <[email protected]>

* clean up node modules and one file that we think is not needed

Signed-off-by: Yee Hing Tong <[email protected]>

---------

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>

* fix: upgrade release node version (#707)

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: update chalk, add semantic-release test cmd (#708)

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* chore: implement logic to handle multiple keys for nested data classes

Signed-off-by: 4nalog <[email protected]>

* chore: fix update_npmversion (#709)

* chore: fix update_npmversion

Signed-off-by: Carina Ursu <[email protected]>

* chore: edits

Signed-off-by: Carina Ursu <[email protected]>

---------

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* Mapped Tasks not showing cache status correctly. (#712)

* fix: cache status logs indexing mismatch and react rendering cycle issue

Signed-off-by: James <[email protected]>

* fix: mapped task cache status

Signed-off-by: James <[email protected]>

---------

Signed-off-by: James <[email protected]>
Co-authored-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: cleanup, passthrough runtime variables (#710)

fix: passthrough runtime variables

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: update_npmversion (#713)

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: fix sed makefile error (#714)

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* FE: Update flyteconsole to Node 18 (#717)

fix: nodejs 18 upgrade

Signed-off-by: James <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* chore: allow complex workflow names (#715)

* chore: allow complex workflow names

Signed-off-by: Carina Ursu <[email protected]>

* chore: fix buold:watch

Signed-off-by: Carina Ursu <[email protected]>

* chore: yarn.lock

Signed-off-by: Carina Ursu <[email protected]>

* chore: upgrade package

Signed-off-by: Carina Ursu <[email protected]>

---------

Signed-off-by: Carina Ursu <[email protected]>

* chore: show correct app version in info  (#716)

* chore: show correct app version in info

Signed-off-by: Carina Ursu <[email protected]>

* chore: update ver

Signed-off-by: Carina Ursu <[email protected]>

---------

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: left nav doesn't accurately update on  workflow version page (#718)

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* feat: differentiate between cache disabled and cache put failure (#719)

fix: add icon for put failure

Signed-off-by: James <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* chore: fix formatting

Signed-off-by: 4nalog <[email protected]>

* fix: add material-ui class name seed (#721)

* chore: add material-ui class name seed

Signed-off-by: Carina Ursu <[email protected]>

* fix: overflow of modal

Signed-off-by: James <[email protected]>

---------

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: James <[email protected]>
Co-authored-by: James <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* The rendering of node status in a dynamic workflow is not functioning correctly (#723)

fix: dynamic nodes status

Signed-off-by: James <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: backfill index on execution task logs (#725)

* chore: backfill index on execution task logs

Signed-off-by: Carina Ursu <[email protected]>

* chore: add comments

Signed-off-by: Carina Ursu <[email protected]>

* chore: bump version

Signed-off-by: Carina Ursu <[email protected]>

* chore: remove backfill

Signed-off-by: Carina Ursu <[email protected]>

* chore: lint

Signed-off-by: Carina Ursu <[email protected]>

---------

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: release fail (#726)

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* Install deps directly in checks.yml (#728)

Signed-off-by: eduardo apolinario <[email protected]>
Co-authored-by: eduardo apolinario <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: show correct i/o in details panel  (#727)

* chore: show correct i/o in details panel

Signed-off-by: Carina Ursu <[email protected]>

* chore: bump version

Signed-off-by: Carina Ursu <[email protected]>

* chore: feedback

Signed-off-by: Carina Ursu <[email protected]>

---------

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* chore: task observability (#720)

* chore: tlro basic setup

Signed-off-by: Carina Ursu <[email protected]>

* progress checkin

* Stable state refactored to use generic naming

* updated types

* working before updating routes

* chore: progress

Signed-off-by: Carina Ursu <[email protected]>

* chore: remove debugger

Signed-off-by: Carina Ursu <[email protected]>

* chore: allow custom class in execution actions

Signed-off-by: Carina Ursu <[email protected]>

* chore: lint fix

Signed-off-by: Carina Ursu <[email protected]>

* chore: bump console

Signed-off-by: Carina Ursu <[email protected]>

* chore: up package version

Signed-off-by: Carina Ursu <[email protected]>

---------

Signed-off-by: Carina Ursu <[email protected]>
Co-authored-by: Jason Porter <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* chore: fix contextual menu action buttons color (#730)

* chore: fix contextual menu action buttons color

Signed-off-by: Carina Ursu <[email protected]>

* chore: up console version

Signed-off-by: Carina Ursu <[email protected]>

---------

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* chore: add build:watch to all packages (#731)

Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* Add REJECT support for ApprovedCondition for GateNodes (#733)

* fix: approveCondition reject

Signed-off-by: James <[email protected]>

* fix: approve button

Signed-off-by: James <[email protected]>

---------

Signed-off-by: James <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* gate node in dynamic task (#729)

* fix: gate node in dynamic

Signed-off-by: James <[email protected]>

* fix: upgraded version

Signed-off-by: James <[email protected]>

* fix: compiledNode in PausedTasksComponent and ExecutionDetailsActions

Signed-off-by: James <[email protected]>

* fix: upgrade version

Signed-off-by: James <[email protected]>

* chore: lockfile

Signed-off-by: Carina Ursu <[email protected]>

* fix: remove caching for workflow closure

Signed-off-by: James <[email protected]>

* fix: break link between cache and context state

Signed-off-by: James <[email protected]>

---------

Signed-off-by: James <[email protected]>
Signed-off-by: Carina Ursu <[email protected]>
Co-authored-by: Carina Ursu <[email protected]>
Signed-off-by: 4nalog <[email protected]>

* fix: comments

Signed-off-by: 4nalog <[email protected]>

---------

Signed-off-by: 4nalog <[email protected]>
Signed-off-by: James <[email protected]>
Signed-off-by: Jason Porter <[email protected]>
Signed-off-by: Carina Ursu <[email protected]>
Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: eduardo apolinario <[email protected]>
Co-authored-by: james-union <[email protected]>
Co-authored-by: Jason Porter <[email protected]>
Co-authored-by: Carina Ursu <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>
Co-authored-by: James <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: eduardo apolinario <[email protected]>
Co-authored-by: Jason Porter <[email protected]>
@flyte-bot
Copy link
Collaborator

🎉 This PR is included in version 1.5.4 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] [CONSOLE] gate node in dynamic task
4 participants