feat: add POST endpoints for validators and validator_balances #6655

nflaig · 2024-04-09T23:10:23Z

Motivation

POST endpoints for validators and validator_balances ethereum/beacon-APIs#367

Description

Adds POST endpoints for validators and validator_balances
More robust handling of numbers in getStateValidatorIndex

Closes #6058

Depends on #6645

codecov · 2024-04-09T23:22:48Z

Codecov Report

Merging #6655 (3c9b04c) into unstable (5ccae1c) will increase coverage by 0.17%.
The diff coverage is 97.77%.

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #6655      +/-   ##
============================================
+ Coverage     61.64%   61.81%   +0.17%     
============================================
  Files           556      556              
  Lines         58782    59050     +268     
  Branches       1886     1898      +12     
============================================
+ Hits          36236    36502     +266     
  Misses        22505    22505              
- Partials         41       43       +2

jeluard · 2024-04-11T12:38:55Z

packages/api/src/beacon/routes/beacon/state.ts

+      }),
+      parseReq: ({params, body}) => [
+        params.state_id,
+        body?.map((id) => (typeof id === "string" && id.startsWith("0x") ? id : fromU64Str(id))),


id is always a string? fromU64Str only accepts string.

I don't think so, if we use string for validator index inside Lodestar it is likely unintended and should be fixed. And even though the type of fromU64Str states in only accepts strings (which imo it should), in practice it will handle numbers as type number just fine.

It only has to be a stringified number over the wire

apis/beacon/states/validators.yaml#L115-L120

ids: type: array uniqueItems: true items: description: "Either hex encoded public key (any bytes48 with 0x prefix) or validator index" type: string # <--

Indices store strictly uses type number

lodestar/packages/validator/src/services/indices.ts

Lines 42 to 45 in 5ccae1c

export class IndicesService {

readonly index2pubkey = new Map<ValidatorIndex, PubkeyHex>();

/** Indexed by pubkey in hex 0x prefixed */

readonly pubkey2index = new Map<PubkeyHex, ValidatorIndex>();

packages/api/src/beacon/routes/beacon/state.ts

jeluard · 2024-04-11T12:46:43Z

packages/beacon-node/src/api/impl/beacon/state/utils.ts

+  }
+
+  if (typeof id === "number") {
+    validatorIndex = id;


Having 2 declaration for validatorIndex with smaller scope would improve readability and prevent the issue you fixed.

The issue was really there because of a wrong type cast, so by removing that it should not be possible anymore. But your suggestion makes sense, simpler to follow state of validatorIndex, as it really doesn't need to be declared at the top of the function

also added unit test to make sure we would catch any regression

packages/beacon-node/src/api/impl/beacon/state/utils.ts

github-actions · 2024-04-11T12:59:16Z

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite	Current: `c2326f1`	Previous: `5ccae1c`	Ratio
Map set x1000	18.222 ns/op	56.321 ns/op	0.32
Set add up to 64 items then delete first	1.6542 us/op	5.4720 us/op	0.30

Full benchmark results

Benchmark suite	Current: `c2326f1`	Previous: `5ccae1c`	Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	378.13 us/op	842.34 us/op	0.45
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	51.803 us/op	81.774 us/op	0.63
BLS verify - blst-native	1.0704 ms/op	1.3026 ms/op	0.82
BLS verifyMultipleSignatures 3 - blst-native	2.2924 ms/op	2.7106 ms/op	0.85
BLS verifyMultipleSignatures 8 - blst-native	5.0912 ms/op	5.9283 ms/op	0.86
BLS verifyMultipleSignatures 32 - blst-native	18.320 ms/op	21.579 ms/op	0.85
BLS verifyMultipleSignatures 64 - blst-native	36.317 ms/op	42.952 ms/op	0.85
BLS verifyMultipleSignatures 128 - blst-native	72.126 ms/op	85.724 ms/op	0.84
BLS deserializing 10000 signatures	783.95 ms/op	871.17 ms/op	0.90
BLS deserializing 100000 signatures	8.1302 s/op	9.3481 s/op	0.87
BLS verifyMultipleSignatures - same message - 3 - blst-native	1.1352 ms/op	1.2953 ms/op	0.88
BLS verifyMultipleSignatures - same message - 8 - blst-native	1.2839 ms/op	1.4560 ms/op	0.88
BLS verifyMultipleSignatures - same message - 32 - blst-native	2.0175 ms/op	2.7478 ms/op	0.73
BLS verifyMultipleSignatures - same message - 64 - blst-native	3.7529 ms/op	4.2240 ms/op	0.89
BLS verifyMultipleSignatures - same message - 128 - blst-native	5.0109 ms/op	7.1888 ms/op	0.70
BLS aggregatePubkeys 32 - blst-native	22.350 us/op	26.605 us/op	0.84
BLS aggregatePubkeys 128 - blst-native	87.787 us/op	97.286 us/op	0.90
notSeenSlots=1 numMissedVotes=1 numBadVotes=10	41.008 ms/op	58.582 ms/op	0.70
notSeenSlots=1 numMissedVotes=0 numBadVotes=4	49.220 ms/op	61.278 ms/op	0.80
notSeenSlots=2 numMissedVotes=1 numBadVotes=10	26.208 ms/op	35.832 ms/op	0.73
getSlashingsAndExits - default max	152.35 us/op	180.65 us/op	0.84
getSlashingsAndExits - 2k	642.85 us/op	400.11 us/op	1.61
proposeBlockBody type=full, size=empty	4.1989 ms/op	5.6710 ms/op	0.74
isKnown best case - 1 super set check	356.00 ns/op	320.00 ns/op	1.11
isKnown normal case - 2 super set checks	368.00 ns/op	321.00 ns/op	1.15
isKnown worse case - 16 super set checks	353.00 ns/op	310.00 ns/op	1.14
InMemoryCheckpointStateCache - add get delete	4.2370 us/op	5.9500 us/op	0.71
validate api signedAggregateAndProof - struct	2.3181 ms/op	2.7793 ms/op	0.83
validate gossip signedAggregateAndProof - struct	2.2987 ms/op	2.7545 ms/op	0.83
validate gossip attestation - vc 640000	1.1462 ms/op	1.3762 ms/op	0.83
batch validate gossip attestation - vc 640000 - chunk 32	147.58 us/op	187.24 us/op	0.79
batch validate gossip attestation - vc 640000 - chunk 64	132.41 us/op	174.01 us/op	0.76
batch validate gossip attestation - vc 640000 - chunk 128	147.30 us/op	167.44 us/op	0.88
batch validate gossip attestation - vc 640000 - chunk 256	114.22 us/op	152.05 us/op	0.75
pickEth1Vote - no votes	934.26 us/op	1.4720 ms/op	0.63
pickEth1Vote - max votes	5.5005 ms/op	11.710 ms/op	0.47
pickEth1Vote - Eth1Data hashTreeRoot value x2048	12.194 ms/op	23.748 ms/op	0.51
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	16.771 ms/op	38.931 ms/op	0.43
pickEth1Vote - Eth1Data fastSerialize value x2048	476.82 us/op	850.61 us/op	0.56
pickEth1Vote - Eth1Data fastSerialize tree x2048	5.9874 ms/op	7.0402 ms/op	0.85
bytes32 toHexString	473.00 ns/op	759.00 ns/op	0.62
bytes32 Buffer.toString(hex)	338.00 ns/op	367.00 ns/op	0.92
bytes32 Buffer.toString(hex) from Uint8Array	523.00 ns/op	596.00 ns/op	0.88
bytes32 Buffer.toString(hex) + 0x	346.00 ns/op	357.00 ns/op	0.97
Object access 1 prop	0.21200 ns/op	0.25700 ns/op	0.82
Map access 1 prop	0.19800 ns/op	0.16900 ns/op	1.17
Object get x1000	5.5520 ns/op	8.3220 ns/op	0.67
Map get x1000	0.88000 ns/op	0.90200 ns/op	0.98
Object set x1000	26.623 ns/op	71.964 ns/op	0.37
Map set x1000	18.222 ns/op	56.321 ns/op	0.32
Return object 10000 times	0.23830 ns/op	0.28320 ns/op	0.84
Throw Error 10000 times	2.8526 us/op	3.9249 us/op	0.73
fastMsgIdFn sha256 / 200 bytes	2.0980 us/op	3.5870 us/op	0.58
fastMsgIdFn h32 xxhash / 200 bytes	361.00 ns/op	376.00 ns/op	0.96
fastMsgIdFn h64 xxhash / 200 bytes	387.00 ns/op	454.00 ns/op	0.85
fastMsgIdFn sha256 / 1000 bytes	6.5120 us/op	11.980 us/op	0.54
fastMsgIdFn h32 xxhash / 1000 bytes	480.00 ns/op	507.00 ns/op	0.95
fastMsgIdFn h64 xxhash / 1000 bytes	488.00 ns/op	493.00 ns/op	0.99
fastMsgIdFn sha256 / 10000 bytes	54.630 us/op	109.46 us/op	0.50
fastMsgIdFn h32 xxhash / 10000 bytes	1.9100 us/op	2.1020 us/op	0.91
fastMsgIdFn h64 xxhash / 10000 bytes	1.3410 us/op	1.4420 us/op	0.93
send data - 1000 256B messages	14.537 ms/op	18.336 ms/op	0.79
send data - 1000 512B messages	18.267 ms/op	30.994 ms/op	0.59
send data - 1000 1024B messages	30.217 ms/op	41.166 ms/op	0.73
send data - 1000 1200B messages	32.895 ms/op	42.436 ms/op	0.78
send data - 1000 2048B messages	34.490 ms/op	54.628 ms/op	0.63
send data - 1000 4096B messages	36.527 ms/op	42.504 ms/op	0.86
send data - 1000 16384B messages	87.338 ms/op	121.58 ms/op	0.72
send data - 1000 65536B messages	398.74 ms/op	466.94 ms/op	0.85
enrSubnets - fastDeserialize 64 bits	1.0700 us/op	1.7220 us/op	0.62
enrSubnets - ssz BitVector 64 bits	447.00 ns/op	558.00 ns/op	0.80
enrSubnets - fastDeserialize 4 bits	224.00 ns/op	247.00 ns/op	0.91
enrSubnets - ssz BitVector 4 bits	492.00 ns/op	539.00 ns/op	0.91
prioritizePeers score -10:0 att 32-0.1 sync 2-0	152.30 us/op	243.18 us/op	0.63
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25	200.29 us/op	323.13 us/op	0.62
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5	344.02 us/op	398.86 us/op	0.86
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75	388.41 us/op	657.41 us/op	0.59
prioritizePeers score 0:0 att 64-1 sync 4-1	509.61 us/op	843.98 us/op	0.60
array of 16000 items push then shift	1.3106 us/op	1.8282 us/op	0.72
LinkedList of 16000 items push then shift	7.3780 ns/op	10.022 ns/op	0.74
array of 16000 items push then pop	83.560 ns/op	112.32 ns/op	0.74
LinkedList of 16000 items push then pop	5.6730 ns/op	9.5540 ns/op	0.59
array of 24000 items push then shift	1.8929 us/op	2.7518 us/op	0.69
LinkedList of 24000 items push then shift	5.9040 ns/op	10.180 ns/op	0.58
array of 24000 items push then pop	92.235 ns/op	166.93 ns/op	0.55
LinkedList of 24000 items push then pop	5.8610 ns/op	9.7310 ns/op	0.60
intersect bitArray bitLen 8	4.9370 ns/op	6.1840 ns/op	0.80
intersect array and set length 8	58.567 ns/op	99.696 ns/op	0.59
intersect bitArray bitLen 128	29.249 ns/op	39.456 ns/op	0.74
intersect array and set length 128	757.08 ns/op	1.2213 us/op	0.62
bitArray.getTrueBitIndexes() bitLen 128	1.2190 us/op	1.9710 us/op	0.62
bitArray.getTrueBitIndexes() bitLen 248	2.0880 us/op	3.8000 us/op	0.55
bitArray.getTrueBitIndexes() bitLen 512	4.4310 us/op	7.7180 us/op	0.57
Buffer.concat 32 items	1.0940 us/op	1.3010 us/op	0.84
Uint8Array.set 32 items	1.7530 us/op	2.6220 us/op	0.67
Set add up to 64 items then delete first	1.6542 us/op	5.4720 us/op	0.30
OrderedSet add up to 64 items then delete first	2.5471 us/op	6.8881 us/op	0.37
Set add up to 64 items then delete last	1.9192 us/op	5.6614 us/op	0.34
OrderedSet add up to 64 items then delete last	2.8254 us/op	7.2747 us/op	0.39
Set add up to 64 items then delete middle	1.8697 us/op	5.4072 us/op	0.35
OrderedSet add up to 64 items then delete middle	3.9666 us/op	8.7194 us/op	0.45
Set add up to 128 items then delete first	3.8205 us/op	11.185 us/op	0.34
OrderedSet add up to 128 items then delete first	6.5096 us/op	15.514 us/op	0.42
Set add up to 128 items then delete last	3.7519 us/op	11.159 us/op	0.34
OrderedSet add up to 128 items then delete last	5.6837 us/op	14.508 us/op	0.39
Set add up to 128 items then delete middle	3.8194 us/op	10.948 us/op	0.35
OrderedSet add up to 128 items then delete middle	10.723 us/op	20.976 us/op	0.51
Set add up to 256 items then delete first	7.8230 us/op	22.988 us/op	0.34
OrderedSet add up to 256 items then delete first	12.273 us/op	31.916 us/op	0.38
Set add up to 256 items then delete last	7.5877 us/op	22.172 us/op	0.34
OrderedSet add up to 256 items then delete last	11.491 us/op	29.506 us/op	0.39
Set add up to 256 items then delete middle	7.6875 us/op	22.327 us/op	0.34
OrderedSet add up to 256 items then delete middle	34.834 us/op	53.999 us/op	0.65
transfer serialized Status (84 B)	1.7520 us/op	2.1080 us/op	0.83
copy serialized Status (84 B)	1.3420 us/op	1.5530 us/op	0.86
transfer serialized SignedVoluntaryExit (112 B)	1.7350 us/op	2.1350 us/op	0.81
copy serialized SignedVoluntaryExit (112 B)	1.5410 us/op	1.5450 us/op	1.00
transfer serialized ProposerSlashing (416 B)	3.3080 us/op	2.4290 us/op	1.36
copy serialized ProposerSlashing (416 B)	2.3650 us/op	2.1530 us/op	1.10
transfer serialized Attestation (485 B)	2.2410 us/op	2.4980 us/op	0.90
copy serialized Attestation (485 B)	1.8490 us/op	2.1900 us/op	0.84
transfer serialized AttesterSlashing (33232 B)	2.5530 us/op	2.2110 us/op	1.15
copy serialized AttesterSlashing (33232 B)	6.4610 us/op	12.617 us/op	0.51
transfer serialized Small SignedBeaconBlock (128000 B)	2.4860 us/op	4.1320 us/op	0.60
copy serialized Small SignedBeaconBlock (128000 B)	18.824 us/op	29.661 us/op	0.63
transfer serialized Avg SignedBeaconBlock (200000 B)	2.9020 us/op	5.4710 us/op	0.53
copy serialized Avg SignedBeaconBlock (200000 B)	33.839 us/op	47.753 us/op	0.71
transfer serialized BlobsSidecar (524380 B)	4.8820 us/op	5.9490 us/op	0.82
copy serialized BlobsSidecar (524380 B)	79.920 us/op	146.00 us/op	0.55
transfer serialized Big SignedBeaconBlock (1000000 B)	3.8270 us/op	6.0870 us/op	0.63
copy serialized Big SignedBeaconBlock (1000000 B)	264.38 us/op	302.84 us/op	0.87
pass gossip attestations to forkchoice per slot	2.8427 ms/op	5.0695 ms/op	0.56
forkChoice updateHead vc 100000 bc 64 eq 0	471.00 us/op	762.02 us/op	0.62
forkChoice updateHead vc 600000 bc 64 eq 0	4.7796 ms/op	4.6831 ms/op	1.02
forkChoice updateHead vc 1000000 bc 64 eq 0	5.0531 ms/op	9.1242 ms/op	0.55
forkChoice updateHead vc 600000 bc 320 eq 0	3.2300 ms/op	5.0409 ms/op	0.64
forkChoice updateHead vc 600000 bc 1200 eq 0	2.8221 ms/op	5.0094 ms/op	0.56
forkChoice updateHead vc 600000 bc 7200 eq 0	3.8812 ms/op	6.2432 ms/op	0.62
forkChoice updateHead vc 600000 bc 64 eq 1000	10.185 ms/op	13.251 ms/op	0.77
forkChoice updateHead vc 600000 bc 64 eq 10000	10.055 ms/op	14.724 ms/op	0.68
forkChoice updateHead vc 600000 bc 64 eq 300000	12.861 ms/op	27.688 ms/op	0.46
computeDeltas 500000 validators 300 proto nodes	3.8752 ms/op	7.6893 ms/op	0.50
computeDeltas 500000 validators 1200 proto nodes	3.1629 ms/op	7.8282 ms/op	0.40
computeDeltas 500000 validators 7200 proto nodes	3.0075 ms/op	7.2236 ms/op	0.42
computeDeltas 750000 validators 300 proto nodes	4.9086 ms/op	11.051 ms/op	0.44
computeDeltas 750000 validators 1200 proto nodes	4.7626 ms/op	11.550 ms/op	0.41
computeDeltas 750000 validators 7200 proto nodes	4.5988 ms/op	11.073 ms/op	0.42
computeDeltas 1400000 validators 300 proto nodes	9.2740 ms/op	21.337 ms/op	0.43
computeDeltas 1400000 validators 1200 proto nodes	9.6491 ms/op	19.634 ms/op	0.49
computeDeltas 1400000 validators 7200 proto nodes	10.367 ms/op	20.173 ms/op	0.51
computeDeltas 2100000 validators 300 proto nodes	14.467 ms/op	29.856 ms/op	0.48
computeDeltas 2100000 validators 1200 proto nodes	13.791 ms/op	28.879 ms/op	0.48
computeDeltas 2100000 validators 7200 proto nodes	13.287 ms/op	28.634 ms/op	0.46
altair processAttestation - 250000 vs - 7PWei normalcase	2.5502 ms/op	2.3347 ms/op	1.09
altair processAttestation - 250000 vs - 7PWei worstcase	2.7282 ms/op	3.4971 ms/op	0.78
altair processAttestation - setStatus - 1/6 committees join	111.28 us/op	157.69 us/op	0.71
altair processAttestation - setStatus - 1/3 committees join	209.66 us/op	301.08 us/op	0.70
altair processAttestation - setStatus - 1/2 committees join	348.31 us/op	412.85 us/op	0.84
altair processAttestation - setStatus - 2/3 committees join	417.78 us/op	501.87 us/op	0.83
altair processAttestation - setStatus - 4/5 committees join	552.70 us/op	701.71 us/op	0.79
altair processAttestation - setStatus - 100% committees join	631.73 us/op	826.99 us/op	0.76
altair processBlock - 250000 vs - 7PWei normalcase	8.5836 ms/op	8.6688 ms/op	0.99
altair processBlock - 250000 vs - 7PWei normalcase hashState	33.561 ms/op	37.142 ms/op	0.90
altair processBlock - 250000 vs - 7PWei worstcase	36.750 ms/op	46.414 ms/op	0.79
altair processBlock - 250000 vs - 7PWei worstcase hashState	91.661 ms/op	92.480 ms/op	0.99
phase0 processBlock - 250000 vs - 7PWei normalcase	2.4107 ms/op	2.7049 ms/op	0.89
phase0 processBlock - 250000 vs - 7PWei worstcase	28.682 ms/op	32.013 ms/op	0.90
altair processEth1Data - 250000 vs - 7PWei normalcase	337.49 us/op	660.31 us/op	0.51
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15	14.089 us/op	9.5110 us/op	1.48
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219	73.624 us/op	71.451 us/op	1.03
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42	23.338 us/op	14.849 us/op	1.57
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18	15.654 us/op	17.387 us/op	0.90
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020	163.51 us/op	192.28 us/op	0.85
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777	1.0639 ms/op	1.1438 ms/op	0.93
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384	1.7892 ms/op	1.6277 ms/op	1.10
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384	1.4932 ms/op	1.5716 ms/op	0.95
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384	3.9882 ms/op	3.8604 ms/op	1.03
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384	2.2763 ms/op	2.3468 ms/op	0.97
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384	5.7027 ms/op	4.6973 ms/op	1.21
Tree 40 250000 create	426.41 ms/op	324.36 ms/op	1.31
Tree 40 250000 get(125000)	160.81 ns/op	194.04 ns/op	0.83
Tree 40 250000 set(125000)	796.22 ns/op	977.17 ns/op	0.81
Tree 40 250000 toArray()	28.157 ms/op	20.080 ms/op	1.40
Tree 40 250000 iterate all - toArray() + loop	24.658 ms/op	20.092 ms/op	1.23
Tree 40 250000 iterate all - get(i)	56.105 ms/op	64.712 ms/op	0.87
MutableVector 250000 create	14.012 ms/op	11.500 ms/op	1.22
MutableVector 250000 get(125000)	5.9360 ns/op	6.4410 ns/op	0.92
MutableVector 250000 set(125000)	485.41 ns/op	279.93 ns/op	1.73
MutableVector 250000 toArray()	3.9889 ms/op	3.0997 ms/op	1.29
MutableVector 250000 iterate all - toArray() + loop	3.9004 ms/op	3.5426 ms/op	1.10
MutableVector 250000 iterate all - get(i)	1.4227 ms/op	1.5274 ms/op	0.93
Array 250000 create	3.1061 ms/op	2.8344 ms/op	1.10
Array 250000 clone - spread	1.3522 ms/op	1.1945 ms/op	1.13
Array 250000 get(125000)	1.1300 ns/op	1.0080 ns/op	1.12
Array 250000 set(125000)	1.3180 ns/op	4.1140 ns/op	0.32
Array 250000 iterate all - loop	160.43 us/op	163.29 us/op	0.98
effectiveBalanceIncrements clone Uint8Array 300000	15.524 us/op	26.781 us/op	0.58
effectiveBalanceIncrements clone MutableVector 300000	448.00 ns/op	340.00 ns/op	1.32
effectiveBalanceIncrements rw all Uint8Array 300000	192.18 us/op	195.78 us/op	0.98
effectiveBalanceIncrements rw all MutableVector 300000	69.831 ms/op	78.718 ms/op	0.89
phase0 afterProcessEpoch - 250000 vs - 7PWei	79.638 ms/op	108.65 ms/op	0.73
phase0 beforeProcessEpoch - 250000 vs - 7PWei	48.147 ms/op	52.053 ms/op	0.92
altair processEpoch - mainnet_e81889	412.16 ms/op	486.59 ms/op	0.85
mainnet_e81889 - altair beforeProcessEpoch	90.464 ms/op	82.312 ms/op	1.10
mainnet_e81889 - altair processJustificationAndFinalization	24.685 us/op	14.417 us/op	1.71
mainnet_e81889 - altair processInactivityUpdates	6.7406 ms/op	6.4702 ms/op	1.04
mainnet_e81889 - altair processRewardsAndPenalties	78.060 ms/op	69.910 ms/op	1.12
mainnet_e81889 - altair processRegistryUpdates	3.8760 us/op	2.3840 us/op	1.63
mainnet_e81889 - altair processSlashings	702.00 ns/op	464.00 ns/op	1.51
mainnet_e81889 - altair processEth1DataReset	889.00 ns/op	504.00 ns/op	1.76
mainnet_e81889 - altair processEffectiveBalanceUpdates	1.0504 ms/op	1.3315 ms/op	0.79
mainnet_e81889 - altair processSlashingsReset	3.8610 us/op	4.2100 us/op	0.92
mainnet_e81889 - altair processRandaoMixesReset	6.6700 us/op	4.9590 us/op	1.35
mainnet_e81889 - altair processHistoricalRootsUpdate	874.00 ns/op	823.00 ns/op	1.06
mainnet_e81889 - altair processParticipationFlagUpdates	2.2000 us/op	1.5550 us/op	1.41
mainnet_e81889 - altair processSyncCommitteeUpdates	799.00 ns/op	783.00 ns/op	1.02
mainnet_e81889 - altair afterProcessEpoch	81.420 ms/op	117.95 ms/op	0.69
capella processEpoch - mainnet_e217614	1.6837 s/op	2.0011 s/op	0.84
mainnet_e217614 - capella beforeProcessEpoch	413.20 ms/op	458.64 ms/op	0.90
mainnet_e217614 - capella processJustificationAndFinalization	12.746 us/op	13.732 us/op	0.93
mainnet_e217614 - capella processInactivityUpdates	14.250 ms/op	20.368 ms/op	0.70
mainnet_e217614 - capella processRewardsAndPenalties	401.50 ms/op	504.54 ms/op	0.80
mainnet_e217614 - capella processRegistryUpdates	19.366 us/op	16.390 us/op	1.18
mainnet_e217614 - capella processSlashings	714.00 ns/op	537.00 ns/op	1.33
mainnet_e217614 - capella processEth1DataReset	371.00 ns/op	420.00 ns/op	0.88
mainnet_e217614 - capella processEffectiveBalanceUpdates	3.1338 ms/op	4.3413 ms/op	0.72
mainnet_e217614 - capella processSlashingsReset	2.6160 us/op	2.6920 us/op	0.97
mainnet_e217614 - capella processRandaoMixesReset	3.1970 us/op	4.4290 us/op	0.72
mainnet_e217614 - capella processHistoricalRootsUpdate	692.00 ns/op	652.00 ns/op	1.06
mainnet_e217614 - capella processParticipationFlagUpdates	2.1070 us/op	1.9030 us/op	1.11
mainnet_e217614 - capella afterProcessEpoch	218.06 ms/op	350.26 ms/op	0.62
phase0 processEpoch - mainnet_e58758	374.36 ms/op	467.15 ms/op	0.80
mainnet_e58758 - phase0 beforeProcessEpoch	123.07 ms/op	116.85 ms/op	1.05
mainnet_e58758 - phase0 processJustificationAndFinalization	16.787 us/op	15.564 us/op	1.08
mainnet_e58758 - phase0 processRewardsAndPenalties	47.816 ms/op	51.794 ms/op	0.92
mainnet_e58758 - phase0 processRegistryUpdates	8.6150 us/op	9.3610 us/op	0.92
mainnet_e58758 - phase0 processSlashings	665.00 ns/op	560.00 ns/op	1.19
mainnet_e58758 - phase0 processEth1DataReset	579.00 ns/op	420.00 ns/op	1.38
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	784.20 us/op	1.1011 ms/op	0.71
mainnet_e58758 - phase0 processSlashingsReset	2.7560 us/op	2.8040 us/op	0.98
mainnet_e58758 - phase0 processRandaoMixesReset	3.2310 us/op	3.4460 us/op	0.94
mainnet_e58758 - phase0 processHistoricalRootsUpdate	586.00 ns/op	410.00 ns/op	1.43
mainnet_e58758 - phase0 processParticipationRecordUpdates	3.0350 us/op	3.8420 us/op	0.79
mainnet_e58758 - phase0 afterProcessEpoch	68.064 ms/op	107.78 ms/op	0.63
phase0 processEffectiveBalanceUpdates - 250000 normalcase	965.97 us/op	1.3109 ms/op	0.74
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	1.0740 ms/op	1.4417 ms/op	0.74
altair processInactivityUpdates - 250000 normalcase	28.354 ms/op	22.857 ms/op	1.24
altair processInactivityUpdates - 250000 worstcase	25.549 ms/op	22.820 ms/op	1.12
phase0 processRegistryUpdates - 250000 normalcase	6.1690 us/op	8.9190 us/op	0.69
phase0 processRegistryUpdates - 250000 badcase_full_deposits	449.81 us/op	384.92 us/op	1.17
phase0 processRegistryUpdates - 250000 worstcase 0.5	124.60 ms/op	128.64 ms/op	0.97
altair processRewardsAndPenalties - 250000 normalcase	61.411 ms/op	51.648 ms/op	1.19
altair processRewardsAndPenalties - 250000 worstcase	73.153 ms/op	66.740 ms/op	1.10
phase0 getAttestationDeltas - 250000 normalcase	5.6384 ms/op	9.1945 ms/op	0.61
phase0 getAttestationDeltas - 250000 worstcase	6.8999 ms/op	9.2697 ms/op	0.74
phase0 processSlashings - 250000 worstcase	94.356 us/op	105.61 us/op	0.89
altair processSyncCommitteeUpdates - 250000	110.57 ms/op	154.36 ms/op	0.72
BeaconState.hashTreeRoot - No change	429.00 ns/op	399.00 ns/op	1.08
BeaconState.hashTreeRoot - 1 full validator	113.53 us/op	138.40 us/op	0.82
BeaconState.hashTreeRoot - 32 full validator	1.5690 ms/op	1.3144 ms/op	1.19
BeaconState.hashTreeRoot - 512 full validator	17.665 ms/op	16.212 ms/op	1.09
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	106.24 us/op	179.30 us/op	0.59
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	1.6352 ms/op	1.9074 ms/op	0.86
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	18.197 ms/op	23.656 ms/op	0.77
BeaconState.hashTreeRoot - 1 balances	83.523 us/op	107.32 us/op	0.78
BeaconState.hashTreeRoot - 32 balances	728.79 us/op	970.29 us/op	0.75
BeaconState.hashTreeRoot - 512 balances	7.2620 ms/op	10.646 ms/op	0.68
BeaconState.hashTreeRoot - 250000 balances	167.97 ms/op	184.18 ms/op	0.91
aggregationBits - 2048 els - zipIndexesInBitList	19.406 us/op	30.086 us/op	0.65
byteArrayEquals 32	65.189 ns/op	76.798 ns/op	0.85
Buffer.compare 32	38.955 ns/op	56.469 ns/op	0.69
byteArrayEquals 1024	1.7732 us/op	2.0557 us/op	0.86
Buffer.compare 1024	46.222 ns/op	71.198 ns/op	0.65
byteArrayEquals 16384	28.527 us/op	32.955 us/op	0.87
Buffer.compare 16384	227.47 ns/op	259.51 ns/op	0.88
byteArrayEquals 123687377	212.96 ms/op	247.05 ms/op	0.86
Buffer.compare 123687377	6.5496 ms/op	6.6102 ms/op	0.99
byteArrayEquals 32 - diff last byte	62.755 ns/op	74.718 ns/op	0.84
Buffer.compare 32 - diff last byte	40.388 ns/op	56.348 ns/op	0.72
byteArrayEquals 1024 - diff last byte	1.7645 us/op	2.0388 us/op	0.87
Buffer.compare 1024 - diff last byte	46.120 ns/op	71.683 ns/op	0.64
byteArrayEquals 16384 - diff last byte	28.116 us/op	32.360 us/op	0.87
Buffer.compare 16384 - diff last byte	231.18 ns/op	257.65 ns/op	0.90
byteArrayEquals 123687377 - diff last byte	212.76 ms/op	246.28 ms/op	0.86
Buffer.compare 123687377 - diff last byte	4.9051 ms/op	8.0701 ms/op	0.61
byteArrayEquals 32 - random bytes	4.6130 ns/op	5.3660 ns/op	0.86
Buffer.compare 32 - random bytes	40.880 ns/op	61.907 ns/op	0.66
byteArrayEquals 1024 - random bytes	4.5020 ns/op	5.2440 ns/op	0.86
Buffer.compare 1024 - random bytes	39.732 ns/op	60.847 ns/op	0.65
byteArrayEquals 16384 - random bytes	4.5090 ns/op	5.2600 ns/op	0.86
Buffer.compare 16384 - random bytes	40.765 ns/op	66.902 ns/op	0.61
byteArrayEquals 123687377 - random bytes	8.0500 ns/op	8.9600 ns/op	0.90
Buffer.compare 123687377 - random bytes	42.930 ns/op	71.350 ns/op	0.60
regular array get 100000 times	42.849 us/op	45.441 us/op	0.94
wrappedArray get 100000 times	42.734 us/op	45.223 us/op	0.94
arrayWithProxy get 100000 times	10.263 ms/op	15.437 ms/op	0.66
ssz.Root.equals	57.655 ns/op	55.952 ns/op	1.03
byteArrayEquals	55.256 ns/op	53.623 ns/op	1.03
Buffer.compare	9.7110 ns/op	11.791 ns/op	0.82
shuffle list - 16384 els	5.5680 ms/op	8.9517 ms/op	0.62
shuffle list - 250000 els	80.900 ms/op	128.81 ms/op	0.63
processSlot - 1 slots	15.995 us/op	15.343 us/op	1.04
processSlot - 32 slots	3.7133 ms/op	3.0698 ms/op	1.21
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	50.619 ms/op	55.973 ms/op	0.90
getCommitteeAssignments - req 1 vs - 250000 vc	2.4277 ms/op	2.6760 ms/op	0.91
getCommitteeAssignments - req 100 vs - 250000 vc	3.6831 ms/op	3.8599 ms/op	0.95
getCommitteeAssignments - req 1000 vs - 250000 vc	3.9698 ms/op	4.2532 ms/op	0.93
findModifiedValidators - 10000 modified validators	307.73 ms/op	381.24 ms/op	0.81
findModifiedValidators - 1000 modified validators	175.81 ms/op	206.55 ms/op	0.85
findModifiedValidators - 100 modified validators	167.88 ms/op	179.86 ms/op	0.93
findModifiedValidators - 10 modified validators	159.26 ms/op	191.25 ms/op	0.83
findModifiedValidators - 1 modified validators	165.92 ms/op	190.57 ms/op	0.87
findModifiedValidators - no difference	200.57 ms/op	210.36 ms/op	0.95
compare ViewDUs	4.2470 s/op	4.6739 s/op	0.91
compare each validator Uint8Array	1.2499 s/op	1.7027 s/op	0.73
compare ViewDU to Uint8Array	1.1886 s/op	1.2766 s/op	0.93
migrate state 1000000 validators, 24 modified, 0 new	738.93 ms/op	762.30 ms/op	0.97
migrate state 1000000 validators, 1700 modified, 1000 new	1.0371 s/op	1.0501 s/op	0.99
migrate state 1000000 validators, 3400 modified, 2000 new	1.2770 s/op	1.3126 s/op	0.97
migrate state 1500000 validators, 24 modified, 0 new	749.95 ms/op	813.79 ms/op	0.92
migrate state 1500000 validators, 1700 modified, 1000 new	974.17 ms/op	1.0772 s/op	0.90
migrate state 1500000 validators, 3400 modified, 2000 new	1.2715 s/op	1.3023 s/op	0.98
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei	5.1700 ns/op	5.0100 ns/op	1.03
state getBlockRootAtSlot - 250000 vs - 7PWei	886.49 ns/op	918.29 ns/op	0.97
computeProposers - vc 250000	9.3215 ms/op	10.276 ms/op	0.91
computeEpochShuffling - vc 250000	92.849 ms/op	131.98 ms/op	0.70
getNextSyncCommittee - vc 250000	110.59 ms/op	174.98 ms/op	0.63
computeSigningRoot for AttestationData	25.513 us/op	27.442 us/op	0.93
hash AttestationData serialized data then Buffer.toString(base64)	1.3925 us/op	2.4209 us/op	0.58
toHexString serialized data	832.53 ns/op	1.1132 us/op	0.75
Buffer.toString(base64)	176.07 ns/op	216.90 ns/op	0.81

by benchmarkbot/action

wemeetagain · 2024-04-23T19:03:32Z

🎉 This PR is included in v1.18.0 🎉

* Add config route definitions * Add debug route definitions * Add events route description * Add lightclient route definitions * Flatten function params * Type safety for optional params in write / parse req * Method args are optional if only optional props * Fix genesisValidatorsRoot type issue * Revert requiring all params in write / parse req * Update http client errors * Add lodestar route definitions * Add node route definitions * Add proof route definitions * Add builder route definitions * Add validator route definitions * Application method response can be void * Generic options can be passed to application methods * Default endpoint request type has body property * Improve types of transform methods * Export server types from index (to be removed) * Update config api impl * Update lightclient api impl * Update events api impl * Update lodestar api impl * Update proof api impl * Update node api impl * Update debug api impl * Update state api impl * Update pool api impl * Update blocks api impl * Partially update validator api impl * Update beacon routes export * Align submitPoolBlsToExecutionChange method args * Filters are always a object * Update errors messages * Add beacon client methods * Add missing routeId label to stream time metric * Fix json casing in codecs * Apply remaining changes from #6227 * Produce block apis only have version meta * Add block values meta to all produce block apis * Apply changes from #6337 * Handle unsafe version in WithMeta and WithVersion * Restore server api error * Update fastify route types * Update server routes / handlers * Remove unnecessary type cast * Restore per route clients * Fix beacon route types * Remove option to patch fetch from http client * Update eventstream client, remove fetch override Fallback does not work like this, see #6180 for proper solution * Use StringType for validator status until #6059 * Remove empty fetch.ts file * Add a few todos * Update builder client and server routes * Update beacon exports * Update api index exports * Update builder index imports * Improve type safety of schema definitions * Add headers to fastify schema * Fix schema definition type * Add missing schemas to route definitions * Fix response codec type * Remove response codec type casts * Fix casing in json only codec * Reuse EmptyResponseCodec * Update base rest api server * Update keymanager routes, client and server * Reuse data types in keymanager impl * Do not await setting headers, not a promise * Improve type safety of empty codecs * Only require to implement supported req methods * Handle requests that only support one format * Handle responses that only support one format * Add json / ssz only req codecs * Update only support errors * Fix assertion * Set correct accept header if only supports one format * Fix eslint / prettier issues * More formatting fixes * Fix fallback request retries in case of http errors * Formatting of res.error * Add add retry functionality to http client (from #6387) * Update rewards routes and server (#6178 and #6260) * Allow to omit body in ssz req if not defined * Always set metadata headers in response * Cache wire format in api response * Only call raw body for json meta * Update api package tests (wip) * Test json and ssz format in generic server tests * Add a bunch of todos * Fix a few broken route definitions * Fix partial config test * Another todo note * Stringify body of json requests * Override default response json method * Validate external consensus version headers in request * Add error handling todo * Skip body schema validation for ssz request bodies * Clean up generic server tests * Pass node, proof, events generic tests * Use enum for media types * Fix a bunch of route definitions * Add justified to blockid type * Properly handle booleans, remove block values codec * Create Uint8Array test data without allocating Buffer * Let fastify handle Buffer conversion * Convert Buffer to Uint8Array in content type parser * Fix build issues * Fix fork type in builder routes * Add some notes * Properly parse request headers * Fix incorrect type assumptions in transform * Generic server tests are passing (except lightclient) * Correctly handle APIs with empty responses * Update getHeader return type to reflect no bid responses * Do not append '?' to URL if query string is empty * Let server handler set status code for parsing errors * Remove unused import * Rename function, request specific * Completely drop ssz support from getSpec * Spec tests are passing against latest releases * Drop unused fastify route config * Drop ssz request from builder routes, not yet supported * Remove import * Apply change from #6695 * Update execution optimistic meta * Apply changes from #6645 * Add workaround to fix epoch committees type issue * Add todo to fix inefficient state conversion * Convert committee to normal array * Apply changes from #6655 * Align args of validators endpoints * Convert indices to str in rewards apis * Update api spec version of README badges * Revert table formatting changes * Make this accessible for class-basd API implementations * Throw err if metadata is accessed for failed response * Add assertOk to api response * Tweak api error message * Update operationIds match spec value * Add missing version to blob sidecars metadata * Test headers and ssz bodies against spec * Minor reordering of code in spec parsing * submitBlindedBlock throws err if fork is not execution * responseOk might be undefined * Remove statusOk from route definition * Remove stale comment * Less build errors in beacon-node * getBlobSidecars return version from server impl * Update validator produce block impl * More expressive pool method args * Application methods might be undefined in mock implementations * Adress open TODOs in server handler * Api response methods are synchronous now * Fix all remaining build issues * Use more performant from/toHex in server api impls * Clean up some TODOs * Fix ApiError type * Errors related to parsing return a 400 status code * Simplify method binding * Forward api context to application methods * There is no easy way to make generic opts typesafe * Better separation of server / client code * Fix comment about missing builder bid * Remove todo, not worth the change / extra indentation * Rename route definitions functions * Return 400 if data passed to keymanager is invalid * Properly handle response metadata headers * Fix lint issues * Add header jsdoc * Move metadata related code into separate file * Remove ssz from POST requests without body * Only set content-type header if body exists * Fix headers extra * POST requests without body are handled similar to GET requests * Fix http client options tests * Improve validation and type safety of JSON metadata * Add type guard for request without body * Differentiate based on body instead of GET vs POST * More renaming * Simplify RequestCode type * Review routes, improve validation * Remaining local diff * Fix accept header handling if only support one wire format * Update 406 error to more closely match spec example * Enforce version header via custom check instead of schema * Use ssz as default request wire format * Log failure to receive head event to verbose * Do not set default value for context * Update getClient return type to better align with method name * Consistent pattern to get route definitions * Dedupe api client type for builder and keymanager * Fix fallback logic if server returns http error * Update head event error logging * Retry 415 errors with JSON and cache SSZ not supported * Use fetch spy to assert call times * Update comment * Update getLightClientUpdatesByRange endpoint meta * Do not forward ssz bytes of blinded block to publishBlock * Fix lightclient e2e tests * Version header in publishBlock api is optional * Reduce type duplication * Add option to override request init in route definition * Add JsonOnlyResp codec * Validate boolean str value from headers * Document default wire formats * Simplify merging of inits in http client * Remove type hacks from fetchBeaconHealth * Reduce call stack in http client * Add .ssz() equivalent method for json to api response * More http client tests * Ensure topics query is provided to eventstream api * Validate request content type in handler Fastify does not cover all edge cases * Review routes, fix param docs, no empty comments * Fix typo * Add note about builder spec not supporting ssz * Consistently move keymanager jsdoc to routes * Sanitize user provided init values before merging * Remove unused ssz only codec * Allow passing wire formats as string literals * chore: review proof routes (#6843) Review proof routes * chore: review lightclient routes (#6842) Review lightclient routes * chore: review node routes (#6844) Review node routes * feat: add cli flags to configure http wire format (#6840) * Review PR, mostly cosmetic changes * Fix event stream error handling --------- Co-authored-by: Cayman <[email protected]>

nflaig added this to the v1.18.0 milestone Apr 9, 2024

Base automatically changed from nflaig/finalized-property to unstable April 11, 2024 10:39

nflaig force-pushed the nflaig/post-endpoints branch 2 times, most recently from cdc92b2 to 39f226d Compare April 11, 2024 12:00

feat: add POST endpoints for validators and validator_balances

a4c6048

nflaig force-pushed the nflaig/post-endpoints branch from 39f226d to a4c6048 Compare April 11, 2024 12:14

nflaig marked this pull request as ready for review April 11, 2024 12:15

nflaig requested a review from a team as a code owner April 11, 2024 12:15

jeluard reviewed Apr 11, 2024

View reviewed changes

Update getStateValidatorIndex

3c9b04c

jeluard self-requested a review April 11, 2024 16:10

jeluard approved these changes Apr 11, 2024

View reviewed changes

nflaig merged commit 669239b into unstable Apr 11, 2024
19 of 20 checks passed

nflaig deleted the nflaig/post-endpoints branch April 11, 2024 17:41

nflaig mentioned this pull request Apr 12, 2024

Cross-client Teku VC Lodestar BN is not working Consensys/teku#8118

Closed

nflaig added a commit that referenced this pull request Apr 28, 2024

Apply changes from #6655

8218615

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add POST endpoints for validators and validator_balances #6655

feat: add POST endpoints for validators and validator_balances #6655

nflaig commented Apr 9, 2024

codecov bot commented Apr 9, 2024 •

edited

Loading

jeluard Apr 11, 2024

nflaig Apr 11, 2024

nflaig Apr 11, 2024 •

edited

Loading

jeluard Apr 11, 2024

nflaig Apr 11, 2024 •

edited

Loading

nflaig Apr 11, 2024

github-actions bot commented Apr 11, 2024 •

edited

Loading

wemeetagain commented Apr 23, 2024

	export class IndicesService {
	readonly index2pubkey = new Map<ValidatorIndex, PubkeyHex>();
	/** Indexed by pubkey in hex 0x prefixed */
	readonly pubkey2index = new Map<PubkeyHex, ValidatorIndex>();

feat: add POST endpoints for validators and validator_balances #6655

feat: add POST endpoints for validators and validator_balances #6655

Conversation

nflaig commented Apr 9, 2024

codecov bot commented Apr 9, 2024 • edited Loading

Codecov Report

jeluard Apr 11, 2024

Choose a reason for hiding this comment

nflaig Apr 11, 2024

Choose a reason for hiding this comment

nflaig Apr 11, 2024 • edited Loading

Choose a reason for hiding this comment

jeluard Apr 11, 2024

Choose a reason for hiding this comment

nflaig Apr 11, 2024 • edited Loading

Choose a reason for hiding this comment

nflaig Apr 11, 2024

Choose a reason for hiding this comment

github-actions bot commented Apr 11, 2024 • edited Loading

Performance Report

wemeetagain commented Apr 23, 2024

codecov bot commented Apr 9, 2024 •

edited

Loading

nflaig Apr 11, 2024 •

edited

Loading

nflaig Apr 11, 2024 •

edited

Loading

github-actions bot commented Apr 11, 2024 •

edited

Loading