Replies: 1 comment 1 reply
-
Fluvio doesn't use DAG tasks which don't seem to be appropriate for the Rust-based stream-based arch. Instead, we use a dependency graph (declarative) to drive scheduling. The data partitioning is a first-class citizen. It is built into topic metadata. The mapping is done implicitly, and affinity policy is pretty simple but we could but more complex mapping using SmartModule as well. |
Beta Was this translation helpful? Give feedback.
1 reply
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
DAG tasks are common in fact, how does fluvio process these tasks? Does fluvio partition data or smartmodule according to the dependency of tasks? And then schedule the smartmodule by considering the affinity about the location of data and dependency.
Beta Was this translation helpful? Give feedback.
All reactions