Benchmarking

[schronck]

Description

We should research the benchmarking features of Substrate and use the weights generated by the benchmarking system in the pallets instead of some hardcoded values.

Solution

• do some research on weights and benchmarking
• write benchmarks for the template and guild pallets
• use a weight template
• use the weights from the benchmark output instead of hardcoded values

[PopcornPaws]

Open a PR to main and let's discuss the results.

[gyozosz]

Some notes on weight

• One unit of weight is supposed to represent one picosecond of execution time on reference hardware
• Setting the reference hardware is a trade-off between performance and the cost of joining to the chain as a validator
• Weights should be calculated by benchmarking the reference hardware
• Substrate's benchmarking assumes linear correlation between the inputs and the execution time of the call so if any calls perform superlinearly then it should not provide a good estimate
• Weights are used to pace the block creation time to be close to the target (6 seconds by default but only 2 seconds are allocated for performing extrinsic calls)
• They can be used to manage the time it takes to validate a block
• Provided weights should represent the worst case scenario on expected hardware
• Transaction fees are based on weights but in case of cheaper execution excess fees are returned
• The blockchain is able to calculate how many transactions or system level calls it can execute within a certain period of time based on the provided weights
• Setting invalid weights may result in overweight blocks which are not secure (See Polkadot paper section 6.5.5)

Sources:

• Substrate docs: glossary
• Substrate docs: fees and weights
• Substrate docs: benchmarking
• Substrate seminar on benchmarking

Note

It's better to set a weight to represent the worst-case scenario (i.e. weights should be greater than the average execution weight)

[PopcornPaws]

Nice summary, thanks!