feat(engine)!: syncing published templates #1222
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…h-template-as-substate
…h-template-as-substate # Conflicts: # dan_layer/consensus/src/consensus_constants.rs # dan_layer/consensus_tests/src/support/harness.rs # dan_layer/engine/src/transaction/error.rs # dan_layer/engine/src/transaction/processor.rs
… template only when block is committed
…h-template-as-substate
…h-template-as-substate # Conflicts: # applications/tari_dan_app_utilities/src/base_layer_scanner.rs # dan_layer/engine_types/src/substate.rs
…h-template-as-substate
…tate' into feature/published-template-sync
…tate' into feature/published-template-sync
…tate' into feature/publish-template-as-substate
…tate' into feature/published-template-sync
…hed-template-sync # Conflicts: # dan_layer/consensus/src/hotstuff/worker.rs
sdbondi
reviewed
Dec 23, 2024
| info!(target: LOG_TARGET, "Template manager initializing"); | ||
| // Template manager | ||
| let (tx_template_sync, rx_template_sync) = broadcast::channel::<TemplateSyncRequest>(100); |
There was a problem hiding this comment.
use an unbounded mpsc channel. Broadcast channel is not appropriate as it is typically used for events and if there are more than 100 requests some of them may start to be dropped (broadcast channel lagging)
sdbondi
reviewed
Dec 23, 2024
| use tari_engine_types::substate::SubstateId; | ||
|
|
||
| /// General trait for different state stores to do operations on substates. | ||
| pub trait SubstateStore { |
There was a problem hiding this comment.
This doesnt seem right. Firstly, this is in the SQLite implementation of the state store which will be a problem when we switch to rocksDB. Templates can always be version 0, so not sure this is necessary, I'll need to look more closely at the rest of the code. There is currently no special process for upgrading the template code, you simply publish a new template at v0.
sdbondi
reviewed
Dec 23, 2024
| } | ||
| if let Some(TemplateResult::Template(fetched_template)) = self.fetch_template(address).optional()? { | ||
| template = Some(fetched_template); | ||
| } |
sdbondi
reviewed
Dec 23, 2024
| template = Some(fetched_template); | ||
| } | ||
| // sleeping here to not overload the local database while waiting for the template to be ready | ||
| thread::sleep(std::time::Duration::from_millis(100)); |
There was a problem hiding this comment.
Not ideal. Will think about how we should handle pending templates in a transaction.
3 tasks
sdbondi
added a commit
that referenced
this pull request
Jan 6, 2025
…llet (#1228) Description --- fix(transactions)!: fix transaction signatures malleability feat(wallet): add support for publishing templates feat(consensus): publish templates to all shards feat(wallet): add UI for publishing templates fix(consensus): correct input involvement info in evidence Wallet daemon now has a transactions.publish_template JSON-RPC call Motivation and Context --- Currently all signatures but one could be surreptitiously removed from the transaction - this PR fixes this by allowing a pre-agreed signer to "seal" the transaction. Wallet daemon now has a `transactions.publish_template` JSON-RPC call. Transactions containing publish template commands are considered "global" i.e. involve all shard groups. This provides the following benefits: 1. All validators immediately have the templates without having to sync them (#1222) and handle the associated edge cases 2. The proof that a template was previously committed is provided by existing mechanisms and consensus 3. Simple to implement Downside is the extra communication overhead of a global transaction  Next steps: - a syncing node does not currently sync templates - syncing the entire template set should be done before the node registers and joins the network and reconciled with synced substates. - remove the actual binary BLOB from the on-chain substate (binary is currently stored twice) - this is currently used when adding the published template to the template store. Instead, one could obtain the BLOB from the transaction instead of from the substate. - swarm should publish templates using the wallet daemon - remove template web URL downloading from base layer UTXO, ignore base layer template registration UTXO How Has This Been Tested? --- Manually, publishing templates through the wallet UI on a multi-shardgroup network. What process can a PR reviewer use to test or verify this change? --- As above Breaking Changes --- - [ ] None - [x] Requires data directory to be deleted - [x] Other - Please specify BREAKING CHANGE: wire-protocol for transactions, database is not compatible, submit transaction JSON-RPC call is not backward-compatible.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Motivation and Context
How Has This Been Tested?
What process can a PR reviewer use to test or verify this change?
Breaking Changes