[Core feature] Fast cache hit success

[hamersaw]

Motivation: Why do you think this is important?

Currently a cached task requires a sequence of state changes to successfully proceed within FlytePropeller. For example, (1) copying inputs from previous nodes, (2) performing cache lookup, (3) writing outputs. Each of these stages requires additional I/O (and time overhead) in committing workflow state at each stage, which may have significant effect on overall workflow performance containing many cached tasks.

Users typically expect cache lookups to be almost instantaneous

Goal: What should the final outcome look like, ideally?

The state change commits required for cache lookups is orthogonal to what users expect. Rather, cache lookups should be almost instantaneous. As a proposal, we can update FlytePropeller to perform cache lookup sequences without stopping to commit node phase updates. This will greatly improve performance and still maintain the correctness guarantees because a node cache lookup is an idempotent operation.

Describe alternatives you've considered

Leaving cache evaluations as is may result in significant workflow evaluation overhead.

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[dylanwilder]

+1 this would be awesome! what's the status of this?

[hamersaw]

+1 this would be awesome! what's the status of this?

@dylanwilder not going to be ready for 1.6, but planning on merging for 1.7. Just making sure it syncs with the ongoing ArrayNode work. Another benefit here, that I know you'll be interested in, is that this moves cache lookups to the node layer, making it possible to support caching of launchplans and subworkflows.