Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: implement peerDAS on electra #6353

Draft
wants to merge 57 commits into
base: unstable
Choose a base branch
from
Draft

feat: implement peerDAS on electra #6353

wants to merge 57 commits into from
+5,561 −4,779

Conversation

g11tech
Copy link
Contributor

@g11tech g11tech commented Jan 24, 2024

@ensi321 ensi321 mentioned this pull request Mar 14, 2024
Open
12 tasks
@g11tech g11tech force-pushed the electra-fork branch 2 times, most recently from 8a4f66e to 473a04f Compare April 11, 2024 11:12
@g11tech g11tech changed the base branch from electra-fork to unstable June 22, 2024 11:16
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 22, 2024 •
edited
Loading

⚠️ Performance Alert ⚠️

Possible performance regression was detected for some benchmarks.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold.

Benchmark suite Current: 39992a5 Previous: 64fe1db Ratio
Array.fill - length 1000000 8.5178 ms/op 2.7179 ms/op 3.13
Full benchmark results
Benchmark suite Current: 39992a5 Previous: 64fe1db Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.1976 ms/op 1.8003 ms/op 1.22
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 65.098 us/op 62.923 us/op 1.03
BLS verify - blst 1.0040 ms/op 856.31 us/op 1.17
BLS verifyMultipleSignatures 3 - blst 1.3275 ms/op 1.3452 ms/op 0.99
BLS verifyMultipleSignatures 8 - blst 2.1431 ms/op 2.1245 ms/op 1.01
BLS verifyMultipleSignatures 32 - blst 5.8520 ms/op 4.6889 ms/op 1.25
BLS verifyMultipleSignatures 64 - blst 9.3095 ms/op 8.5742 ms/op 1.09
BLS verifyMultipleSignatures 128 - blst 17.852 ms/op 16.264 ms/op 1.10
BLS deserializing 10000 signatures 707.54 ms/op 636.46 ms/op 1.11
BLS deserializing 100000 signatures 7.1271 s/op 6.3021 s/op 1.13
BLS verifyMultipleSignatures - same message - 3 - blst 1.1379 ms/op 1.0099 ms/op 1.13
BLS verifyMultipleSignatures - same message - 8 - blst 1.2073 ms/op 1.0873 ms/op 1.11
BLS verifyMultipleSignatures - same message - 32 - blst 1.8035 ms/op 1.7250 ms/op 1.05
BLS verifyMultipleSignatures - same message - 64 - blst 2.6714 ms/op 2.5609 ms/op 1.04
BLS verifyMultipleSignatures - same message - 128 - blst 4.6866 ms/op 4.1836 ms/op 1.12
BLS aggregatePubkeys 32 - blst 21.485 us/op 18.196 us/op 1.18
BLS aggregatePubkeys 128 - blst 72.274 us/op 64.035 us/op 1.13
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 65.850 ms/op 65.789 ms/op 1.00
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 61.685 ms/op 47.840 ms/op 1.29
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 41.640 ms/op 32.076 ms/op 1.30
getSlashingsAndExits - default max 109.74 us/op 69.649 us/op 1.58
getSlashingsAndExits - 2k 427.97 us/op 263.28 us/op 1.63
proposeBlockBody type=full, size=empty 6.8063 ms/op 5.2151 ms/op 1.31
isKnown best case - 1 super set check 638.00 ns/op 492.00 ns/op 1.30
isKnown normal case - 2 super set checks 612.00 ns/op 495.00 ns/op 1.24
isKnown worse case - 16 super set checks 640.00 ns/op 474.00 ns/op 1.35
InMemoryCheckpointStateCache - add get delete 7.7490 us/op 4.5260 us/op 1.71
validate api signedAggregateAndProof - struct 2.0130 ms/op 1.7464 ms/op 1.15
validate gossip signedAggregateAndProof - struct 1.6346 ms/op 1.5010 ms/op 1.09
validate gossip attestation - vc 640000 1.1903 ms/op 1.0377 ms/op 1.15
batch validate gossip attestation - vc 640000 - chunk 32 165.14 us/op 132.93 us/op 1.24
batch validate gossip attestation - vc 640000 - chunk 64 145.62 us/op 114.51 us/op 1.27
batch validate gossip attestation - vc 640000 - chunk 128 133.60 us/op 112.24 us/op 1.19
batch validate gossip attestation - vc 640000 - chunk 256 121.39 us/op 111.62 us/op 1.09
pickEth1Vote - no votes 1.2966 ms/op 1.0398 ms/op 1.25
pickEth1Vote - max votes 12.980 ms/op 11.599 ms/op 1.12
pickEth1Vote - Eth1Data hashTreeRoot value x2048 17.115 ms/op 20.416 ms/op 0.84
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 23.001 ms/op 22.145 ms/op 1.04
pickEth1Vote - Eth1Data fastSerialize value x2048 586.99 us/op 597.66 us/op 0.98
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.9056 ms/op 4.4597 ms/op 1.10
bytes32 toHexString 603.00 ns/op 840.00 ns/op 0.72
bytes32 Buffer.toString(hex) 266.00 ns/op 494.00 ns/op 0.54
bytes32 Buffer.toString(hex) from Uint8Array 470.00 ns/op 591.00 ns/op 0.80
bytes32 Buffer.toString(hex) + 0x 299.00 ns/op 479.00 ns/op 0.62
Object access 1 prop 0.18800 ns/op 0.36000 ns/op 0.52
Map access 1 prop 0.14600 ns/op 0.38100 ns/op 0.38
Object get x1000 6.0730 ns/op 5.2660 ns/op 1.15
Map get x1000 6.5440 ns/op 5.9960 ns/op 1.09
Object set x1000 47.212 ns/op 30.727 ns/op 1.54
Map set x1000 33.268 ns/op 26.693 ns/op 1.25
Return object 10000 times 0.31470 ns/op 0.32890 ns/op 0.96
Throw Error 10000 times 3.5493 us/op 3.1036 us/op 1.14
fastMsgIdFn sha256 / 200 bytes 2.4630 us/op 2.1860 us/op 1.13
fastMsgIdFn h32 xxhash / 200 bytes 299.00 ns/op 527.00 ns/op 0.57
fastMsgIdFn h64 xxhash / 200 bytes 291.00 ns/op 509.00 ns/op 0.57
fastMsgIdFn sha256 / 1000 bytes 8.0940 us/op 6.3000 us/op 1.28
fastMsgIdFn h32 xxhash / 1000 bytes 436.00 ns/op 654.00 ns/op 0.67
fastMsgIdFn h64 xxhash / 1000 bytes 394.00 ns/op 570.00 ns/op 0.69
fastMsgIdFn sha256 / 10000 bytes 66.174 us/op 52.322 us/op 1.26
fastMsgIdFn h32 xxhash / 10000 bytes 1.9760 us/op 2.0890 us/op 0.95
fastMsgIdFn h64 xxhash / 10000 bytes 1.2860 us/op 1.3170 us/op 0.98
send data - 1000 256B messages 15.065 ms/op 12.966 ms/op 1.16
send data - 1000 512B messages 20.708 ms/op 19.483 ms/op 1.06
send data - 1000 1024B messages 28.600 ms/op 25.797 ms/op 1.11
send data - 1000 1200B messages 29.976 ms/op 25.429 ms/op 1.18
send data - 1000 2048B messages 35.116 ms/op 37.352 ms/op 0.94
send data - 1000 4096B messages 33.935 ms/op 33.505 ms/op 1.01
send data - 1000 16384B messages 93.286 ms/op 68.949 ms/op 1.35
send data - 1000 65536B messages 223.61 ms/op 286.88 ms/op 0.78
enrSubnets - fastDeserialize 64 bits 1.3100 us/op 1.2740 us/op 1.03
enrSubnets - ssz BitVector 64 bits 441.00 ns/op 631.00 ns/op 0.70
enrSubnets - fastDeserialize 4 bits 213.00 ns/op 383.00 ns/op 0.56
enrSubnets - ssz BitVector 4 bits 527.00 ns/op 630.00 ns/op 0.84
prioritizePeers score -10:0 att 32-0.1 sync 2-0 220.07 us/op 118.18 us/op 1.86
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 190.52 us/op 131.81 us/op 1.45
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 412.11 us/op 232.95 us/op 1.77
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 500.47 us/op 505.56 us/op 0.99
prioritizePeers score 0:0 att 64-1 sync 4-1 876.81 us/op 739.96 us/op 1.18
array of 16000 items push then shift 1.7268 us/op 1.4520 us/op 1.19
LinkedList of 16000 items push then shift 8.1690 ns/op 6.8030 ns/op 1.20
array of 16000 items push then pop 135.62 ns/op 120.13 ns/op 1.13
LinkedList of 16000 items push then pop 7.6270 ns/op 6.3790 ns/op 1.20
array of 24000 items push then shift 2.4659 us/op 1.9434 us/op 1.27
LinkedList of 24000 items push then shift 7.8000 ns/op 6.9240 ns/op 1.13
array of 24000 items push then pop 182.22 ns/op 132.65 ns/op 1.37
LinkedList of 24000 items push then pop 7.6880 ns/op 6.4250 ns/op 1.20
intersect bitArray bitLen 8 6.9940 ns/op 5.6920 ns/op 1.23
intersect array and set length 8 59.324 ns/op 42.036 ns/op 1.41
intersect bitArray bitLen 128 30.119 ns/op 26.879 ns/op 1.12
intersect array and set length 128 930.85 ns/op 607.83 ns/op 1.53
bitArray.getTrueBitIndexes() bitLen 128 2.8130 us/op 2.4000 us/op 1.17
bitArray.getTrueBitIndexes() bitLen 248 4.5010 us/op 3.5530 us/op 1.27
bitArray.getTrueBitIndexes() bitLen 512 10.378 us/op 8.8160 us/op 1.18
Buffer.concat 32 items 1.1120 us/op 1.1880 us/op 0.94
Uint8Array.set 32 items 1.6110 us/op 1.7700 us/op 0.91
Buffer.copy 1.7600 us/op 2.3830 us/op 0.74
Uint8Array.set - with subarray 2.7550 us/op 3.2580 us/op 0.85
Uint8Array.set - without subarray 1.6380 us/op 1.9690 us/op 0.83
getUint32 - dataview 250.00 ns/op 515.00 ns/op 0.49
getUint32 - manual 212.00 ns/op 416.00 ns/op 0.51
Set add up to 64 items then delete first 2.3608 us/op 1.8459 us/op 1.28
OrderedSet add up to 64 items then delete first 3.4199 us/op 2.9095 us/op 1.18
Set add up to 64 items then delete last 2.9062 us/op 2.0943 us/op 1.39
OrderedSet add up to 64 items then delete last 4.1422 us/op 3.1956 us/op 1.30
Set add up to 64 items then delete middle 3.0099 us/op 2.0910 us/op 1.44
OrderedSet add up to 64 items then delete middle 6.4745 us/op 4.6970 us/op 1.38
Set add up to 128 items then delete first 6.3620 us/op 4.2091 us/op 1.51
OrderedSet add up to 128 items then delete first 9.8281 us/op 6.4835 us/op 1.52
Set add up to 128 items then delete last 6.2509 us/op 4.1051 us/op 1.52
OrderedSet add up to 128 items then delete last 10.069 us/op 6.1582 us/op 1.64
Set add up to 128 items then delete middle 6.6312 us/op 4.0013 us/op 1.66
OrderedSet add up to 128 items then delete middle 16.400 us/op 11.806 us/op 1.39
Set add up to 256 items then delete first 13.276 us/op 8.0984 us/op 1.64
OrderedSet add up to 256 items then delete first 22.344 us/op 12.644 us/op 1.77
Set add up to 256 items then delete last 13.373 us/op 7.8239 us/op 1.71
OrderedSet add up to 256 items then delete last 19.944 us/op 12.012 us/op 1.66
Set add up to 256 items then delete middle 12.928 us/op 7.8280 us/op 1.65
OrderedSet add up to 256 items then delete middle 48.062 us/op 34.473 us/op 1.39
transfer serialized Status (84 B) 1.4620 us/op 1.3820 us/op 1.06
copy serialized Status (84 B) 1.3340 us/op 1.1740 us/op 1.14
transfer serialized SignedVoluntaryExit (112 B) 1.5380 us/op 1.4620 us/op 1.05
copy serialized SignedVoluntaryExit (112 B) 1.3140 us/op 1.2060 us/op 1.09
transfer serialized ProposerSlashing (416 B) 2.3200 us/op 1.5520 us/op 1.49
copy serialized ProposerSlashing (416 B) 2.2160 us/op 1.3760 us/op 1.61
transfer serialized Attestation (485 B) 2.0410 us/op 1.5580 us/op 1.31
copy serialized Attestation (485 B) 2.0720 us/op 1.4470 us/op 1.43
transfer serialized AttesterSlashing (33232 B) 2.7110 us/op 1.7540 us/op 1.55
copy serialized AttesterSlashing (33232 B) 9.3580 us/op 3.7900 us/op 2.47
transfer serialized Small SignedBeaconBlock (128000 B) 2.8550 us/op 2.2270 us/op 1.28
copy serialized Small SignedBeaconBlock (128000 B) 26.490 us/op 9.4900 us/op 2.79
transfer serialized Avg SignedBeaconBlock (200000 B) 3.6920 us/op 2.3660 us/op 1.56
copy serialized Avg SignedBeaconBlock (200000 B) 36.103 us/op 13.623 us/op 2.65
transfer serialized BlobsSidecar (524380 B) 4.7710 us/op 2.9430 us/op 1.62
copy serialized BlobsSidecar (524380 B) 116.18 us/op 75.550 us/op 1.54
transfer serialized Big SignedBeaconBlock (1000000 B) 4.5440 us/op 2.9160 us/op 1.56
copy serialized Big SignedBeaconBlock (1000000 B) 166.11 us/op 154.13 us/op 1.08
pass gossip attestations to forkchoice per slot 3.3956 ms/op 2.7926 ms/op 1.22
forkChoice updateHead vc 100000 bc 64 eq 0 571.29 us/op 426.91 us/op 1.34
forkChoice updateHead vc 600000 bc 64 eq 0 3.2541 ms/op 2.5391 ms/op 1.28
forkChoice updateHead vc 1000000 bc 64 eq 0 5.6838 ms/op 4.3560 ms/op 1.30
forkChoice updateHead vc 600000 bc 320 eq 0 3.2372 ms/op 2.6182 ms/op 1.24
forkChoice updateHead vc 600000 bc 1200 eq 0 3.2667 ms/op 2.7149 ms/op 1.20
forkChoice updateHead vc 600000 bc 7200 eq 0 4.2432 ms/op 3.7385 ms/op 1.14
forkChoice updateHead vc 600000 bc 64 eq 1000 11.015 ms/op 9.6213 ms/op 1.14
forkChoice updateHead vc 600000 bc 64 eq 10000 11.162 ms/op 9.4834 ms/op 1.18
forkChoice updateHead vc 600000 bc 64 eq 300000 18.266 ms/op 11.980 ms/op 1.52
computeDeltas 500000 validators 300 proto nodes 4.0408 ms/op 3.0627 ms/op 1.32
computeDeltas 500000 validators 1200 proto nodes 4.1414 ms/op 3.0399 ms/op 1.36
computeDeltas 500000 validators 7200 proto nodes 4.0764 ms/op 2.9911 ms/op 1.36
computeDeltas 750000 validators 300 proto nodes 5.6888 ms/op 4.7289 ms/op 1.20
computeDeltas 750000 validators 1200 proto nodes 6.4143 ms/op 4.6347 ms/op 1.38
computeDeltas 750000 validators 7200 proto nodes 6.1108 ms/op 4.6151 ms/op 1.32
computeDeltas 1400000 validators 300 proto nodes 10.817 ms/op 8.3119 ms/op 1.30
computeDeltas 1400000 validators 1200 proto nodes 13.310 ms/op 8.1609 ms/op 1.63
computeDeltas 1400000 validators 7200 proto nodes 12.054 ms/op 8.2134 ms/op 1.47
computeDeltas 2100000 validators 300 proto nodes 17.815 ms/op 12.282 ms/op 1.45
computeDeltas 2100000 validators 1200 proto nodes 22.415 ms/op 12.577 ms/op 1.78
computeDeltas 2100000 validators 7200 proto nodes 20.812 ms/op 12.910 ms/op 1.61
altair processAttestation - 250000 vs - 7PWei normalcase 3.2413 ms/op 1.5401 ms/op 2.10
altair processAttestation - 250000 vs - 7PWei worstcase 4.4385 ms/op 2.2254 ms/op 1.99
altair processAttestation - setStatus - 1/6 committees join 135.00 us/op 74.810 us/op 1.80
altair processAttestation - setStatus - 1/3 committees join 315.37 us/op 142.85 us/op 2.21
altair processAttestation - setStatus - 1/2 committees join 362.29 us/op 216.25 us/op 1.68
altair processAttestation - setStatus - 2/3 committees join 480.43 us/op 273.63 us/op 1.76
altair processAttestation - setStatus - 4/5 committees join 569.89 us/op 394.40 us/op 1.44
altair processAttestation - setStatus - 100% committees join 666.83 us/op 467.08 us/op 1.43
altair processBlock - 250000 vs - 7PWei normalcase 5.6000 ms/op 3.1173 ms/op 1.80
altair processBlock - 250000 vs - 7PWei normalcase hashState 35.576 ms/op 20.148 ms/op 1.77
altair processBlock - 250000 vs - 7PWei worstcase 49.812 ms/op 35.478 ms/op 1.40
altair processBlock - 250000 vs - 7PWei worstcase hashState 113.05 ms/op 70.610 ms/op 1.60
phase0 processBlock - 250000 vs - 7PWei normalcase 3.0941 ms/op 1.9545 ms/op 1.58
phase0 processBlock - 250000 vs - 7PWei worstcase 32.327 ms/op 25.429 ms/op 1.27
altair processEth1Data - 250000 vs - 7PWei normalcase 562.62 us/op 247.95 us/op 2.27
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 12.326 us/op 6.0540 us/op 2.04
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 46.360 us/op 16.441 us/op 2.82
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 17.023 us/op 8.2150 us/op 2.07
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 12.086 us/op 5.7670 us/op 2.10
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 159.96 us/op 85.160 us/op 1.88
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 946.50 us/op 662.79 us/op 1.43
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.5918 ms/op 1.1798 ms/op 1.35
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.6846 ms/op 652.58 us/op 2.58
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.6305 ms/op 2.0388 ms/op 1.78
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.0357 ms/op 1.2019 ms/op 1.69
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.1531 ms/op 3.1490 ms/op 1.64
Tree 40 250000 create 524.18 ms/op 194.30 ms/op 2.70
Tree 40 250000 get(125000) 177.92 ns/op 149.67 ns/op 1.19
Tree 40 250000 set(125000) 1.8321 us/op 634.88 ns/op 2.89
Tree 40 250000 toArray() 28.183 ms/op 12.248 ms/op 2.30
Tree 40 250000 iterate all - toArray() + loop 25.942 ms/op 11.756 ms/op 2.21
Tree 40 250000 iterate all - get(i) 71.923 ms/op 47.107 ms/op 1.53
MutableVector 250000 create 13.124 ms/op 17.297 ms/op 0.76
MutableVector 250000 get(125000) 6.9220 ns/op 6.3670 ns/op 1.09
MutableVector 250000 set(125000) 697.90 ns/op 368.82 ns/op 1.89
MutableVector 250000 toArray() 5.7485 ms/op 5.3246 ms/op 1.08
MutableVector 250000 iterate all - toArray() + loop 5.3201 ms/op 4.2869 ms/op 1.24
MutableVector 250000 iterate all - get(i) 1.8181 ms/op 1.5370 ms/op 1.18
Array 250000 create 4.5246 ms/op 3.3638 ms/op 1.35
Array 250000 clone - spread 2.0668 ms/op 1.2673 ms/op 1.63
Array 250000 get(125000) 0.54300 ns/op 0.61800 ns/op 0.88
Array 250000 set(125000) 0.60400 ns/op 0.65400 ns/op 0.92
Array 250000 iterate all - loop 115.50 us/op 77.773 us/op 1.49
effectiveBalanceIncrements clone Uint8Array 300000 71.172 us/op 61.961 us/op 1.15
effectiveBalanceIncrements clone MutableVector 300000 184.00 ns/op 334.00 ns/op 0.55
effectiveBalanceIncrements rw all Uint8Array 300000 265.20 us/op 171.20 us/op 1.55
effectiveBalanceIncrements rw all MutableVector 300000 179.85 ms/op 63.577 ms/op 2.83
phase0 afterProcessEpoch - 250000 vs - 7PWei 116.82 ms/op 78.179 ms/op 1.49
Array.fill - length 1000000 8.5178 ms/op 2.7179 ms/op 3.13
Array push - length 1000000 32.688 ms/op 19.537 ms/op 1.67
Array.get 0.40148 ns/op 0.31950 ns/op 1.26
Uint8Array.get 0.55767 ns/op 0.35816 ns/op 1.56
phase0 beforeProcessEpoch - 250000 vs - 7PWei 26.036 ms/op 23.994 ms/op 1.09
altair processEpoch - mainnet_e81889 456.93 ms/op 380.53 ms/op 1.20
mainnet_e81889 - altair beforeProcessEpoch 24.569 ms/op 23.390 ms/op 1.05
mainnet_e81889 - altair processJustificationAndFinalization 27.256 us/op 22.569 us/op 1.21
mainnet_e81889 - altair processInactivityUpdates 9.7422 ms/op 10.437 ms/op 0.93
mainnet_e81889 - altair processRewardsAndPenalties 66.277 ms/op 69.232 ms/op 0.96
mainnet_e81889 - altair processRegistryUpdates 5.2780 us/op 4.7740 us/op 1.11
mainnet_e81889 - altair processSlashings 1.4170 us/op 1.0420 us/op 1.36
mainnet_e81889 - altair processEth1DataReset 1.2230 us/op 995.00 ns/op 1.23
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.2341 ms/op 2.8388 ms/op 0.79
mainnet_e81889 - altair processSlashingsReset 7.7620 us/op 5.7720 us/op 1.34
mainnet_e81889 - altair processRandaoMixesReset 9.3570 us/op 7.4340 us/op 1.26
mainnet_e81889 - altair processHistoricalRootsUpdate 1.4250 us/op 1.5050 us/op 0.95
mainnet_e81889 - altair processParticipationFlagUpdates 4.9520 us/op 2.0870 us/op 2.37
mainnet_e81889 - altair processSyncCommitteeUpdates 1.0180 us/op 741.00 ns/op 1.37
mainnet_e81889 - altair afterProcessEpoch 118.38 ms/op 75.074 ms/op 1.58
capella processEpoch - mainnet_e217614 1.4603 s/op 1.1964 s/op 1.22
mainnet_e217614 - capella beforeProcessEpoch 90.149 ms/op 73.537 ms/op 1.23
mainnet_e217614 - capella processJustificationAndFinalization 41.089 us/op 13.446 us/op 3.06
mainnet_e217614 - capella processInactivityUpdates 25.604 ms/op 16.759 ms/op 1.53
mainnet_e217614 - capella processRewardsAndPenalties 300.16 ms/op 248.65 ms/op 1.21
mainnet_e217614 - capella processRegistryUpdates 23.223 us/op 11.336 us/op 2.05
mainnet_e217614 - capella processSlashings 815.00 ns/op 866.00 ns/op 0.94
mainnet_e217614 - capella processEth1DataReset 870.00 ns/op 682.00 ns/op 1.28
mainnet_e217614 - capella processEffectiveBalanceUpdates 15.912 ms/op 11.665 ms/op 1.36
mainnet_e217614 - capella processSlashingsReset 8.0520 us/op 2.4620 us/op 3.27
mainnet_e217614 - capella processRandaoMixesReset 10.024 us/op 3.5970 us/op 2.79
mainnet_e217614 - capella processHistoricalRootsUpdate 1.9530 us/op 704.00 ns/op 2.77
mainnet_e217614 - capella processParticipationFlagUpdates 4.3770 us/op 1.8100 us/op 2.42
mainnet_e217614 - capella afterProcessEpoch 271.34 ms/op 196.48 ms/op 1.38
phase0 processEpoch - mainnet_e58758 509.98 ms/op 345.84 ms/op 1.47
mainnet_e58758 - phase0 beforeProcessEpoch 117.14 ms/op 70.299 ms/op 1.67
mainnet_e58758 - phase0 processJustificationAndFinalization 30.024 us/op 12.206 us/op 2.46
mainnet_e58758 - phase0 processRewardsAndPenalties 43.462 ms/op 33.229 ms/op 1.31
mainnet_e58758 - phase0 processRegistryUpdates 12.856 us/op 7.9050 us/op 1.63
mainnet_e58758 - phase0 processSlashings 972.00 ns/op 429.00 ns/op 2.27
mainnet_e58758 - phase0 processEth1DataReset 1.0130 us/op 436.00 ns/op 2.32
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.2733 ms/op 787.93 us/op 1.62
mainnet_e58758 - phase0 processSlashingsReset 5.4560 us/op 5.7780 us/op 0.94
mainnet_e58758 - phase0 processRandaoMixesReset 7.5670 us/op 3.4520 us/op 2.19
mainnet_e58758 - phase0 processHistoricalRootsUpdate 826.00 ns/op 812.00 ns/op 1.02
mainnet_e58758 - phase0 processParticipationRecordUpdates 6.0380 us/op 3.4280 us/op 1.76
mainnet_e58758 - phase0 afterProcessEpoch 88.959 ms/op 61.985 ms/op 1.44
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.4254 ms/op 835.28 us/op 1.71
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.0531 ms/op 1.7232 ms/op 1.19
altair processInactivityUpdates - 250000 normalcase 21.098 ms/op 16.160 ms/op 1.31
altair processInactivityUpdates - 250000 worstcase 20.106 ms/op 17.038 ms/op 1.18
phase0 processRegistryUpdates - 250000 normalcase 7.4530 us/op 6.2790 us/op 1.19
phase0 processRegistryUpdates - 250000 badcase_full_deposits 414.85 us/op 307.56 us/op 1.35
phase0 processRegistryUpdates - 250000 worstcase 0.5 145.39 ms/op 110.64 ms/op 1.31
altair processRewardsAndPenalties - 250000 normalcase 47.771 ms/op 40.407 ms/op 1.18
altair processRewardsAndPenalties - 250000 worstcase 45.939 ms/op 45.614 ms/op 1.01
phase0 getAttestationDeltas - 250000 normalcase 11.700 ms/op 9.7247 ms/op 1.20
phase0 getAttestationDeltas - 250000 worstcase 10.488 ms/op 7.3208 ms/op 1.43
phase0 processSlashings - 250000 worstcase 116.08 us/op 91.465 us/op 1.27
altair processSyncCommitteeUpdates - 250000 140.51 ms/op 93.373 ms/op 1.50
BeaconState.hashTreeRoot - No change 374.00 ns/op 483.00 ns/op 0.77
BeaconState.hashTreeRoot - 1 full validator 105.19 us/op 81.728 us/op 1.29
BeaconState.hashTreeRoot - 32 full validator 1.1364 ms/op 1.1766 ms/op 0.97
BeaconState.hashTreeRoot - 512 full validator 14.536 ms/op 13.484 ms/op 1.08
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 146.59 us/op 130.36 us/op 1.12
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.5559 ms/op 2.2585 ms/op 1.13
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 29.869 ms/op 23.216 ms/op 1.29
BeaconState.hashTreeRoot - 1 balances 108.55 us/op 105.60 us/op 1.03
BeaconState.hashTreeRoot - 32 balances 854.46 us/op 1.0965 ms/op 0.78
BeaconState.hashTreeRoot - 512 balances 9.3056 ms/op 6.0743 ms/op 1.53
BeaconState.hashTreeRoot - 250000 balances 213.42 ms/op 143.85 ms/op 1.48
aggregationBits - 2048 els - zipIndexesInBitList 31.914 us/op 21.135 us/op 1.51
byteArrayEquals 32 55.249 ns/op 49.621 ns/op 1.11
Buffer.compare 32 17.511 ns/op 16.255 ns/op 1.08
byteArrayEquals 1024 1.6270 us/op 1.3090 us/op 1.24
Buffer.compare 1024 26.149 ns/op 24.463 ns/op 1.07
byteArrayEquals 16384 25.986 us/op 20.403 us/op 1.27
Buffer.compare 16384 198.23 ns/op 208.14 ns/op 0.95
byteArrayEquals 123687377 199.35 ms/op 148.76 ms/op 1.34
Buffer.compare 123687377 9.8990 ms/op 4.9662 ms/op 1.99
byteArrayEquals 32 - diff last byte 56.451 ns/op 48.030 ns/op 1.18
Buffer.compare 32 - diff last byte 18.572 ns/op 18.063 ns/op 1.03
byteArrayEquals 1024 - diff last byte 1.7205 us/op 1.2652 us/op 1.36
Buffer.compare 1024 - diff last byte 27.212 ns/op 23.671 ns/op 1.15
byteArrayEquals 16384 - diff last byte 26.735 us/op 20.175 us/op 1.33
Buffer.compare 16384 - diff last byte 187.88 ns/op 209.97 ns/op 0.89
byteArrayEquals 123687377 - diff last byte 196.67 ms/op 151.42 ms/op 1.30
Buffer.compare 123687377 - diff last byte 9.5086 ms/op 4.1535 ms/op 2.29
byteArrayEquals 32 - random bytes 5.9550 ns/op 4.9080 ns/op 1.21
Buffer.compare 32 - random bytes 18.662 ns/op 16.164 ns/op 1.15
byteArrayEquals 1024 - random bytes 5.2880 ns/op 4.9000 ns/op 1.08
Buffer.compare 1024 - random bytes 17.463 ns/op 16.341 ns/op 1.07
byteArrayEquals 16384 - random bytes 5.2880 ns/op 4.9340 ns/op 1.07
Buffer.compare 16384 - random bytes 17.405 ns/op 15.898 ns/op 1.09
byteArrayEquals 123687377 - random bytes 6.5700 ns/op 7.9300 ns/op 0.83
Buffer.compare 123687377 - random bytes 19.310 ns/op 19.980 ns/op 0.97
regular array get 100000 times 35.101 us/op 30.721 us/op 1.14
wrappedArray get 100000 times 33.938 us/op 30.744 us/op 1.10
arrayWithProxy get 100000 times 14.643 ms/op 10.097 ms/op 1.45
ssz.Root.equals 46.890 ns/op 42.737 ns/op 1.10
byteArrayEquals 46.586 ns/op 41.315 ns/op 1.13
Buffer.compare 11.395 ns/op 8.5290 ns/op 1.34
shuffle list - 16384 els 6.4095 ms/op 5.3619 ms/op 1.20
shuffle list - 250000 els 91.905 ms/op 78.890 ms/op 1.16
processSlot - 1 slots 14.973 us/op 13.035 us/op 1.15
processSlot - 32 slots 2.7050 ms/op 2.7752 ms/op 0.97
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 41.293 ms/op 41.244 ms/op 1.00
getCommitteeAssignments - req 1 vs - 250000 vc 2.2194 ms/op 1.8990 ms/op 1.17
getCommitteeAssignments - req 100 vs - 250000 vc 4.3342 ms/op 3.5554 ms/op 1.22
getCommitteeAssignments - req 1000 vs - 250000 vc 4.5757 ms/op 3.8388 ms/op 1.19
findModifiedValidators - 10000 modified validators 380.27 ms/op 242.77 ms/op 1.57
findModifiedValidators - 1000 modified validators 245.55 ms/op 139.23 ms/op 1.76
findModifiedValidators - 100 modified validators 277.37 ms/op 134.75 ms/op 2.06
findModifiedValidators - 10 modified validators 268.67 ms/op 130.98 ms/op 2.05
findModifiedValidators - 1 modified validators 223.47 ms/op 149.37 ms/op 1.50
findModifiedValidators - no difference 204.27 ms/op 160.25 ms/op 1.27
compare ViewDUs 3.6356 s/op 3.0351 s/op 1.20
compare each validator Uint8Array 1.8709 s/op 1.5140 s/op 1.24
compare ViewDU to Uint8Array 1.0601 s/op 651.28 ms/op 1.63
migrate state 1000000 validators, 24 modified, 0 new 613.56 ms/op 524.51 ms/op 1.17
migrate state 1000000 validators, 1700 modified, 1000 new 901.91 ms/op 727.18 ms/op 1.24
migrate state 1000000 validators, 3400 modified, 2000 new 1.2015 s/op 906.29 ms/op 1.33
migrate state 1500000 validators, 24 modified, 0 new 720.14 ms/op 485.80 ms/op 1.48
migrate state 1500000 validators, 1700 modified, 1000 new 970.86 ms/op 816.74 ms/op 1.19
migrate state 1500000 validators, 3400 modified, 2000 new 1.3290 s/op 1.0334 s/op 1.29
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.1400 ns/op 6.0800 ns/op 0.85
state getBlockRootAtSlot - 250000 vs - 7PWei 691.24 ns/op 652.34 ns/op 1.06
computeProposers - vc 250000 8.8279 ms/op 6.1335 ms/op 1.44
computeEpochShuffling - vc 250000 104.15 ms/op 76.014 ms/op 1.37
getNextSyncCommittee - vc 250000 149.75 ms/op 104.99 ms/op 1.43
computeSigningRoot for AttestationData 26.884 us/op 20.805 us/op 1.29
hash AttestationData serialized data then Buffer.toString(base64) 1.7116 us/op 1.1798 us/op 1.45
toHexString serialized data 1.1509 us/op 792.81 ns/op 1.45
Buffer.toString(base64) 226.05 ns/op 132.86 ns/op 1.70

by benchmarkbot/action

@g11tech
feat: placeholder PR for electra
4805a2e 
add types stub and epoch config

fix types
@g11tech
feat: implement peerDAS on electra
499d93c 
add some presets

add further params and types

add data column to types repo and network

move to max request data columns to preset

add the datacolumns data in blockinput and fix breaking errors in seen gossip blockinput

handle data columns in gossip and the seengossip

further propagate forkaware blockdata and resolve build/type issues

further handle datacolumns sync by range by root and forkaware data handling

fix issues

chore: update c-kzg to peerDas version

feat: add peerDas ckzg functions to interface

fix the lookups

handle the publishing flow

various sync try fixes

fixes

compute blob side car

various misl debuggings and fixes

debug and apply fixes and get range and by root sync to work will full custody

enable syncing with lower custody requirement

use node peerid rather than a dummy string

get and use the nodeid from enr and correctly compute subnets and column indexes

filterout and connect to peers only matching out custody requiremnt

try adding custody requirement

add protection for subnet calc

get the sync working with devnet 0

correctly set the enr with custody subnet info

rebase fixes

small refactor
matthewkeil and others added 5 commits August 9, 2024 17:40
@matthewkeil @g11tech
fix: docker build issue for c-kzg
156ef53 
wip: REPLACE THIS COMMIT

commit yarn lock

rebase fixes

fix: update c-zkg install workflow

feat: add trustedSetupPrecompute cli flag

fix: update trusted-setup for testing

fix: update c-zkg install workflow to remove sudo

fix: add rsync to apk deps
@g11tech
feat: get various sync mechanisms working with/without sharded data
47eedae
@g11tech
feat: add the modifications to work with devnet2
d423004 
some network options to control peering behavior

allow setting node custody capability via --params

use eip754 names for the peerdas config
@g11tech
fix: refactor to add and use nodeid computation and clear out nodeid …
a0c5d27 
…tracking

nodeid cleanup for network
@g11tech
fix the types/test
c7f6341 
rebase fixes
@g11tech
feat: add and use metadatav3 for peer custody subnet
81aaeb5 
fixes for metadata, working locally

change the condition to update metadata csc change
g11tech and others added 11 commits September 11, 2024 17:07
@g11tech
add some more loggig and availaibility tracking
f7571f4
@g11tech
add some log for debugging inbound data columns request
74d8122
@g11tech
some fixes
af933fb
@g11tech
custodied column fetch debugging log
56c8c6e
@g11tech
datacolumns retrival fix
2b10e4d
@g11tech
update compute spec tests
cdd9bae
@g11tech
fix the column id compute
c4d04ee
@g11tech
more debug log
3470076
@g11tech
edge case optimization
4ec7aff
@matthewkeil
feat: refactor and unit test getDataColumnSidecars (#7072)
a33303f 
* refactor: getDataColumnSidecars

* test: unit test getDataColumnSidecars with mocks from c-kzg library

* refactor: use fromHex util

* chore: update numbering on mocks

* chore: update c-kzg to latest version

* chore: fix type export syntax

* test: add verification for cells from sidecars

* test: add verification to DataColumnSidecars tests

* refactor: getDataColumnSidecars for PR comments

* feat: narrow type and remove unnecessary conditional

* fix: getDataColumnSidecars param type

* refactor: rename to computeDataColumnSidecars
@matthewkeil
feat: update ckzg to final DAS version (#7050)
bd4f7f9 
* feat: update c-kzg to final DAS version

* refactor: use trusted-setup from c-kzg package
matthewkeil and others added 18 commits September 17, 2024 14:56
@matthewkeil
fix: remove ckzg build script (#7089)
b1940ee 
* fix: remove unused ckzg build script

* fix: remove unused rsync dep from Dockerfile
@matthewkeil
feat: validate data column sidecars (#7073)
20ef4c6 
* feat: update c-kzg to final DAS version

* refactor: use trusted-setup from c-kzg package

* feat: implement validateDataColumnsSidecars

* feat: check block and column commitments match

* test: add unit test for validateDataColumnsSidecars

* fix: invalid build and update validity condition of validateDataColumnsSidecars

* fix: make error messages better

* fix: electra vs peerdas type conflict
@g11tech
validate inclusion proof
fee7c08
@g11tech
use sample subnets for data availability
574837a
@g11tech
add debug console log
6a77828
@g11tech
handle edge case
cec27d6
@g11tech
turn persisting network identity to default true
a3de70f
@g11tech
improve logging for debugging
cce193b
@g11tech
add enhance datacolumn serving logs
2736b8c
@g11tech
more log
a0e0087
@g11tech
migrate datacolumns to finalized
b04aaef
@g11tech
add debug and fix datacolumns migration and improve log
7c9a01c
@g11tech
some fixing of beacon params
1c08ab3
@g11tech
fix
513bccc
@g11tech
add prevdownload tracker
fccf9a2
@matthewkeil
feat: check for no commitments on block or column in sidecar validati…
8689c76 
…on (#7184)

* feat: check for no commitments on block or column in sidecar validation

* test: add sanity check for empty blob commitments in column validation

* fix: test bug

* fix: but in test passing commitments
@matthewkeil
feat: log peer disconnect info (#7231)
c8075d0 
* feat: log disconnect reason

* feat: log peerScore update

* fix: pretty print peerId

* fix: use prettyPrintPeerId
@matthewkeil
refactor: peerdas types (#7243)
e8bc729 
* refactor: organize peerDAS types

* refactor: DataColumnsData

* refactor: rename BlockInputBlobs BlockInputColumnData

* refactor: split up and rename BlockInputData

* refactor: clean up BlobsData

* refactor: clean up CachedData types

* refactor: change from interface to type and update enum values for grafana

* chore: lint

* fix: remove extraneous lint fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matthewkeil matthewkeil matthewkeil left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@g11tech @matthewkeil