Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Housekeeping] Remove barrier transitions from FlytePropeller #3544

Closed
2 tasks done
hamersaw opened this issue Mar 28, 2023 · 0 comments · Fixed by flyteorg/flytepropeller#545
Closed
2 tasks done
Assignees
Labels
housekeeping Issues that help maintain flyte and keep it tech-debt free
Milestone

Comments

@hamersaw
Copy link
Contributor

Describe the issue

Currently, FlytePropeller manages plugin execution by processing a sequence of transitions that are either ephemeral or barrier. The barrier types are meant to ensure that the associated transition is reported before moving to another plugin transition. However, these transitions are stored in an in-memory cache and are therefore lost between restarts. So, they are essentially a very good best effort. It is not clear that these are useful and only result in additional overhead and complexity.

What if we do not do this?

The current situations results in a minimal amount of unnecessary overhead for every execution, inflated memory costs in the barrier cache, and useless complexity in the codebase.

Related component(s)

No response

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@hamersaw hamersaw added the housekeeping Issues that help maintain flyte and keep it tech-debt free label Mar 28, 2023
@hamersaw hamersaw self-assigned this Mar 28, 2023
@hamersaw hamersaw added this to the 1.6.0 milestone Mar 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
housekeeping Issues that help maintain flyte and keep it tech-debt free
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant