Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Synchronize node and operator start #236

Merged
merged 3 commits into from
Mar 24, 2023
Merged

Synchronize node and operator start #236

merged 3 commits into from
Mar 24, 2023

Conversation

phil-opp
Copy link
Collaborator

@phil-opp phil-opp commented Mar 23, 2023 •
edited
Loading

  • Synchronize startup of nodes through subscribe request
    • Nodes wait for a reply after sending a subscribe request.
    • By delaying that reply until all nodes are subscribed, we can synchronize the start time of the dataflow.
  • Only start timer tasks once dataflow is ready to start
    • This way, we don't fill the queue with tick events that occured before the synchronized start happened.
  • dora-runtime: Only subscribe to daemon once all operators are ready
    • The dora-runtime uses the normal node API, so it uses the same synchronization system.
    • By running the DoraNode::init function only after all operators were initialized, we ensure that they are ready when the dataflow starts.

Resolves #233

@phil-opp phil-opp changed the base branch from prevent-queue-buildup to main March 23, 2023 16:50
@phil-opp phil-opp requested a review from haixuanTao March 23, 2023 16:50
@phil-opp phil-opp mentioned this pull request Mar 23, 2023
Closed
5 tasks
haixuanTao
haixuanTao approved these changes Mar 24, 2023
View reviewed changes
Copy link
Collaborator

@haixuanTao haixuanTao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

@haixuanTao haixuanTao merged commit d952e80 into main Mar 24, 2023
@haixuanTao haixuanTao deleted the start-together branch March 24, 2023 02:27
haixuanTao added a commit that referenced this pull request Mar 27, 2023
@haixuanTao
Push error into the init_done channel for debugging context
bfd32f1 
After adding #236 , error of initialization is shadowed by bailing on
the sender channel.

Bailing used to happen here: https://github.com/dora-rs/dora/blob/ccec196234c46fc5df7a0f7c7cd15d29a2bda670/binaries/runtime/src/lib.rs#L136

This makes it hard to debug.

This PR will push the error into the sender channel, and report any error
of initialization making it easier to debug.

Now the channel is used without error, but any error of initialization
will bail the runtime here: https://github.com/dora-rs/dora/blob/ccec196234c46fc5df7a0f7c7cd15d29a2bda670/binaries/runtime/src/lib.rs#L137
@haixuanTao haixuanTao mentioned this pull request Mar 27, 2023
Merged
@phil-opp phil-opp mentioned this pull request Apr 26, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@haixuanTao haixuanTao haixuanTao approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add Event::OutputReady to the Event stream
2 participants
@phil-opp @haixuanTao