Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reuse stripOffNewlines to trim keystore password from file #5283

Merged
merged 3 commits into from
Mar 19, 2023
Merged

Reuse stripOffNewlines to trim keystore password from file #5283

merged 3 commits into from
Mar 19, 2023

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented Mar 19, 2023 •
edited
Loading

Motivation

Follow up PR for #5282 to reuse existing stripOffNewlines function and add some unit tests to ensure it works

Description

  • Fix stripOffNewlines to not remove pipe character
  • Add unit tests to verify stripOffNewlines works as expected
  • Reuse stripOffNewlines to trim keystore password from file

@nflaig nflaig requested a review from a team as a code owner March 19, 2023 09:14
@nflaig nflaig mentioned this pull request Mar 19, 2023
Merged
@github-actions
Copy link
Contributor

github-actions bot commented Mar 19, 2023 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 5a7dcbb Previous: ae4e442 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 544.29 us/op 834.42 us/op 0.65
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 46.816 us/op 46.652 us/op 1.00
BLS verify - blst-native 1.2207 ms/op 1.2070 ms/op 1.01
BLS verifyMultipleSignatures 3 - blst-native 2.4902 ms/op 2.4465 ms/op 1.02
BLS verifyMultipleSignatures 8 - blst-native 5.3374 ms/op 5.2711 ms/op 1.01
BLS verifyMultipleSignatures 32 - blst-native 19.332 ms/op 19.128 ms/op 1.01
BLS aggregatePubkeys 32 - blst-native 25.881 us/op 25.651 us/op 1.01
BLS aggregatePubkeys 128 - blst-native 101.72 us/op 99.721 us/op 1.02
getAttestationsForBlock 54.891 ms/op 52.131 ms/op 1.05
isKnown best case - 1 super set check 254.00 ns/op 247.00 ns/op 1.03
isKnown normal case - 2 super set checks 247.00 ns/op 244.00 ns/op 1.01
isKnown worse case - 16 super set checks 250.00 ns/op 242.00 ns/op 1.03
CheckpointStateCache - add get delete 4.9430 us/op 4.7480 us/op 1.04
validate gossip signedAggregateAndProof - struct 2.7523 ms/op 2.6501 ms/op 1.04
validate gossip attestation - struct 1.3159 ms/op 1.2638 ms/op 1.04
pickEth1Vote - no votes 1.2865 ms/op 1.2041 ms/op 1.07
pickEth1Vote - max votes 10.746 ms/op 10.751 ms/op 1.00
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.6005 ms/op 8.8423 ms/op 0.97
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 14.467 ms/op 14.639 ms/op 0.99
pickEth1Vote - Eth1Data fastSerialize value x2048 650.42 us/op 639.71 us/op 1.02
pickEth1Vote - Eth1Data fastSerialize tree x2048 8.1004 ms/op 7.1611 ms/op 1.13
bytes32 toHexString 494.00 ns/op 471.00 ns/op 1.05
bytes32 Buffer.toString(hex) 323.00 ns/op 327.00 ns/op 0.99
bytes32 Buffer.toString(hex) from Uint8Array 521.00 ns/op 526.00 ns/op 0.99
bytes32 Buffer.toString(hex) + 0x 329.00 ns/op 331.00 ns/op 0.99
Object access 1 prop 0.15700 ns/op 0.15400 ns/op 1.02
Map access 1 prop 0.14800 ns/op 0.15100 ns/op 0.98
Object get x1000 6.1000 ns/op 6.9810 ns/op 0.87
Map get x1000 0.54300 ns/op 0.55600 ns/op 0.98
Object set x1000 50.357 ns/op 49.550 ns/op 1.02
Map set x1000 41.684 ns/op 41.229 ns/op 1.01
Return object 10000 times 0.22440 ns/op 0.22590 ns/op 0.99
Throw Error 10000 times 4.0512 us/op 4.0264 us/op 1.01
fastMsgIdFn sha256 / 200 bytes 3.2790 us/op 3.2960 us/op 0.99
fastMsgIdFn h32 xxhash / 200 bytes 270.00 ns/op 258.00 ns/op 1.05
fastMsgIdFn h64 xxhash / 200 bytes 376.00 ns/op 374.00 ns/op 1.01
fastMsgIdFn sha256 / 1000 bytes 11.151 us/op 11.129 us/op 1.00
fastMsgIdFn h32 xxhash / 1000 bytes 397.00 ns/op 383.00 ns/op 1.04
fastMsgIdFn h64 xxhash / 1000 bytes 450.00 ns/op 442.00 ns/op 1.02
fastMsgIdFn sha256 / 10000 bytes 99.146 us/op 99.888 us/op 0.99
fastMsgIdFn h32 xxhash / 10000 bytes 1.8600 us/op 1.8160 us/op 1.02
fastMsgIdFn h64 xxhash / 10000 bytes 1.3010 us/op 1.2920 us/op 1.01
enrSubnets - fastDeserialize 64 bits 1.2480 us/op 1.2250 us/op 1.02
enrSubnets - ssz BitVector 64 bits 464.00 ns/op 471.00 ns/op 0.99
enrSubnets - fastDeserialize 4 bits 166.00 ns/op 161.00 ns/op 1.03
enrSubnets - ssz BitVector 4 bits 464.00 ns/op 470.00 ns/op 0.99
prioritizePeers score -10:0 att 32-0.1 sync 2-0 98.708 us/op 97.721 us/op 1.01
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 128.68 us/op 127.84 us/op 1.01
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 157.27 us/op 161.13 us/op 0.98
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 286.35 us/op 291.84 us/op 0.98
prioritizePeers score 0:0 att 64-1 sync 4-1 342.13 us/op 353.56 us/op 0.97
array of 16000 items push then shift 1.5529 us/op 1.5873 us/op 0.98
LinkedList of 16000 items push then shift 8.4050 ns/op 8.4790 ns/op 0.99
array of 16000 items push then pop 75.998 ns/op 80.322 ns/op 0.95
LinkedList of 16000 items push then pop 8.1000 ns/op 8.4590 ns/op 0.96
array of 24000 items push then shift 2.2329 us/op 2.3483 us/op 0.95
LinkedList of 24000 items push then shift 8.3410 ns/op 8.3890 ns/op 0.99
array of 24000 items push then pop 71.394 ns/op 73.704 ns/op 0.97
LinkedList of 24000 items push then pop 8.0560 ns/op 8.5010 ns/op 0.95
intersect bitArray bitLen 8 12.674 ns/op 13.219 ns/op 0.96
intersect array and set length 8 73.492 ns/op 77.017 ns/op 0.95
intersect bitArray bitLen 128 41.734 ns/op 44.122 ns/op 0.95
intersect array and set length 128 1.0137 us/op 1.0436 us/op 0.97
Buffer.concat 32 items 2.4640 us/op 2.8910 us/op 0.85
Uint8Array.set 32 items 2.0330 us/op 2.4900 us/op 0.82
pass gossip attestations to forkchoice per slot 2.8362 ms/op 2.7417 ms/op 1.03
computeDeltas 3.3286 ms/op 2.8515 ms/op 1.17
computeProposerBoostScoreFromBalances 1.7776 ms/op 1.7512 ms/op 1.02
altair processAttestation - 250000 vs - 7PWei normalcase 2.4095 ms/op 2.0340 ms/op 1.18
altair processAttestation - 250000 vs - 7PWei worstcase 3.3938 ms/op 3.5390 ms/op 0.96
altair processAttestation - setStatus - 1/6 committees join 145.24 us/op 136.80 us/op 1.06
altair processAttestation - setStatus - 1/3 committees join 277.21 us/op 270.37 us/op 1.03
altair processAttestation - setStatus - 1/2 committees join 369.49 us/op 362.37 us/op 1.02
altair processAttestation - setStatus - 2/3 committees join 463.24 us/op 465.71 us/op 0.99
altair processAttestation - setStatus - 4/5 committees join 655.25 us/op 634.55 us/op 1.03
altair processAttestation - setStatus - 100% committees join 755.93 us/op 764.21 us/op 0.99
altair processBlock - 250000 vs - 7PWei normalcase 15.168 ms/op 15.587 ms/op 0.97
altair processBlock - 250000 vs - 7PWei normalcase hashState 26.652 ms/op 27.532 ms/op 0.97
altair processBlock - 250000 vs - 7PWei worstcase 45.010 ms/op 45.742 ms/op 0.98
altair processBlock - 250000 vs - 7PWei worstcase hashState 72.672 ms/op 66.386 ms/op 1.09
phase0 processBlock - 250000 vs - 7PWei normalcase 2.0407 ms/op 2.1132 ms/op 0.97
phase0 processBlock - 250000 vs - 7PWei worstcase 26.848 ms/op 27.952 ms/op 0.96
altair processEth1Data - 250000 vs - 7PWei normalcase 451.73 us/op 479.21 us/op 0.94
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.8410 us/op 6.7230 us/op 1.02
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 19.656 us/op 19.277 us/op 1.02
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 8.5670 us/op 8.1990 us/op 1.04
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.5840 us/op 6.1230 us/op 1.08
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 76.356 us/op 74.623 us/op 1.02
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 639.94 us/op 608.09 us/op 1.05
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 898.20 us/op 889.84 us/op 1.01
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 835.26 us/op 907.08 us/op 0.92
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.2355 ms/op 2.2625 ms/op 0.99
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.6495 ms/op 1.5759 ms/op 1.05
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.7344 ms/op 3.6061 ms/op 1.04
Tree 40 250000 create 302.70 ms/op 312.68 ms/op 0.97
Tree 40 250000 get(125000) 186.16 ns/op 175.70 ns/op 1.06
Tree 40 250000 set(125000) 873.29 ns/op 856.48 ns/op 1.02
Tree 40 250000 toArray() 16.342 ms/op 16.132 ms/op 1.01
Tree 40 250000 iterate all - toArray() + loop 16.432 ms/op 16.438 ms/op 1.00
Tree 40 250000 iterate all - get(i) 63.247 ms/op 64.811 ms/op 0.98
MutableVector 250000 create 10.507 ms/op 10.087 ms/op 1.04
MutableVector 250000 get(125000) 6.2850 ns/op 6.2720 ns/op 1.00
MutableVector 250000 set(125000) 256.12 ns/op 247.05 ns/op 1.04
MutableVector 250000 toArray() 2.8507 ms/op 2.7164 ms/op 1.05
MutableVector 250000 iterate all - toArray() + loop 2.8659 ms/op 2.7856 ms/op 1.03
MutableVector 250000 iterate all - get(i) 1.5218 ms/op 1.5062 ms/op 1.01
Array 250000 create 2.5510 ms/op 2.5300 ms/op 1.01
Array 250000 clone - spread 1.2716 ms/op 1.1826 ms/op 1.08
Array 250000 get(125000) 0.59800 ns/op 0.57200 ns/op 1.05
Array 250000 set(125000) 0.68100 ns/op 0.65100 ns/op 1.05
Array 250000 iterate all - loop 109.50 us/op 81.253 us/op 1.35
effectiveBalanceIncrements clone Uint8Array 300000 27.344 us/op 31.747 us/op 0.86
effectiveBalanceIncrements clone MutableVector 300000 409.00 ns/op 373.00 ns/op 1.10
effectiveBalanceIncrements rw all Uint8Array 300000 166.85 us/op 165.91 us/op 1.01
effectiveBalanceIncrements rw all MutableVector 300000 82.793 ms/op 81.263 ms/op 1.02
phase0 afterProcessEpoch - 250000 vs - 7PWei 106.28 ms/op 110.67 ms/op 0.96
phase0 beforeProcessEpoch - 250000 vs - 7PWei 34.907 ms/op 32.854 ms/op 1.06
altair processEpoch - mainnet_e81889 294.68 ms/op 329.84 ms/op 0.89
mainnet_e81889 - altair beforeProcessEpoch 63.292 ms/op 65.840 ms/op 0.96
mainnet_e81889 - altair processJustificationAndFinalization 15.095 us/op 16.604 us/op 0.91
mainnet_e81889 - altair processInactivityUpdates 5.0454 ms/op 5.1355 ms/op 0.98
mainnet_e81889 - altair processRewardsAndPenalties 67.551 ms/op 49.382 ms/op 1.37
mainnet_e81889 - altair processRegistryUpdates 2.6910 us/op 2.2680 us/op 1.19
mainnet_e81889 - altair processSlashings 524.00 ns/op 449.00 ns/op 1.17
mainnet_e81889 - altair processEth1DataReset 544.00 ns/op 525.00 ns/op 1.04
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.3444 ms/op 1.2434 ms/op 1.08
mainnet_e81889 - altair processSlashingsReset 4.8620 us/op 4.7260 us/op 1.03
mainnet_e81889 - altair processRandaoMixesReset 4.4870 us/op 4.7450 us/op 0.95
mainnet_e81889 - altair processHistoricalRootsUpdate 568.00 ns/op 534.00 ns/op 1.06
mainnet_e81889 - altair processParticipationFlagUpdates 2.5130 us/op 3.3680 us/op 0.75
mainnet_e81889 - altair processSyncCommitteeUpdates 544.00 ns/op 467.00 ns/op 1.16
mainnet_e81889 - altair afterProcessEpoch 114.38 ms/op 127.16 ms/op 0.90
phase0 processEpoch - mainnet_e58758 314.79 ms/op 361.10 ms/op 0.87
mainnet_e58758 - phase0 beforeProcessEpoch 120.43 ms/op 139.48 ms/op 0.86
mainnet_e58758 - phase0 processJustificationAndFinalization 17.599 us/op 17.369 us/op 1.01
mainnet_e58758 - phase0 processRewardsAndPenalties 52.560 ms/op 62.824 ms/op 0.84
mainnet_e58758 - phase0 processRegistryUpdates 7.7910 us/op 7.5880 us/op 1.03
mainnet_e58758 - phase0 processSlashings 457.00 ns/op 459.00 ns/op 1.00
mainnet_e58758 - phase0 processEth1DataReset 460.00 ns/op 455.00 ns/op 1.01
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 968.26 us/op 1.0121 ms/op 0.96
mainnet_e58758 - phase0 processSlashingsReset 2.7350 us/op 3.3930 us/op 0.81
mainnet_e58758 - phase0 processRandaoMixesReset 4.4690 us/op 4.2190 us/op 1.06
mainnet_e58758 - phase0 processHistoricalRootsUpdate 536.00 ns/op 900.00 ns/op 0.60
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.9930 us/op 4.1250 us/op 0.97
mainnet_e58758 - phase0 afterProcessEpoch 96.291 ms/op 97.181 ms/op 0.99
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2150 ms/op 1.2535 ms/op 0.97
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.3534 ms/op 1.5194 ms/op 0.89
altair processInactivityUpdates - 250000 normalcase 20.757 ms/op 24.675 ms/op 0.84
altair processInactivityUpdates - 250000 worstcase 23.094 ms/op 25.937 ms/op 0.89
phase0 processRegistryUpdates - 250000 normalcase 6.2460 us/op 14.826 us/op 0.42
phase0 processRegistryUpdates - 250000 badcase_full_deposits 240.58 us/op 270.80 us/op 0.89
phase0 processRegistryUpdates - 250000 worstcase 0.5 115.02 ms/op 123.90 ms/op 0.93
altair processRewardsAndPenalties - 250000 normalcase 48.600 ms/op 66.764 ms/op 0.73
altair processRewardsAndPenalties - 250000 worstcase 52.799 ms/op 71.601 ms/op 0.74
phase0 getAttestationDeltas - 250000 normalcase 6.2769 ms/op 6.8957 ms/op 0.91
phase0 getAttestationDeltas - 250000 worstcase 6.3323 ms/op 6.8344 ms/op 0.93
phase0 processSlashings - 250000 worstcase 3.4269 ms/op 3.6577 ms/op 0.94
altair processSyncCommitteeUpdates - 250000 173.11 ms/op 180.02 ms/op 0.96
BeaconState.hashTreeRoot - No change 258.00 ns/op 259.00 ns/op 1.00
BeaconState.hashTreeRoot - 1 full validator 50.025 us/op 56.099 us/op 0.89
BeaconState.hashTreeRoot - 32 full validator 549.60 us/op 503.98 us/op 1.09
BeaconState.hashTreeRoot - 512 full validator 4.9107 ms/op 5.6199 ms/op 0.87
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 60.569 us/op 62.892 us/op 0.96
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 853.49 us/op 932.89 us/op 0.91
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 10.803 ms/op 11.325 ms/op 0.95
BeaconState.hashTreeRoot - 1 balances 46.682 us/op 49.233 us/op 0.95
BeaconState.hashTreeRoot - 32 balances 452.13 us/op 440.77 us/op 1.03
BeaconState.hashTreeRoot - 512 balances 4.3051 ms/op 4.4388 ms/op 0.97
BeaconState.hashTreeRoot - 250000 balances 74.258 ms/op 77.975 ms/op 0.95
aggregationBits - 2048 els - zipIndexesInBitList 16.472 us/op 16.173 us/op 1.02
regular array get 100000 times 41.504 us/op 33.500 us/op 1.24
wrappedArray get 100000 times 31.947 us/op 33.342 us/op 0.96
arrayWithProxy get 100000 times 16.300 ms/op 16.457 ms/op 0.99
ssz.Root.equals 546.00 ns/op 600.00 ns/op 0.91
byteArrayEquals 536.00 ns/op 546.00 ns/op 0.98
shuffle list - 16384 els 6.8376 ms/op 7.0012 ms/op 0.98
shuffle list - 250000 els 99.876 ms/op 102.85 ms/op 0.97
processSlot - 1 slots 8.2800 us/op 8.4470 us/op 0.98
processSlot - 32 slots 1.3218 ms/op 1.3373 ms/op 0.99
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 33.138 ms/op 36.086 ms/op 0.92
getCommitteeAssignments - req 1 vs - 250000 vc 2.8511 ms/op 2.9143 ms/op 0.98
getCommitteeAssignments - req 100 vs - 250000 vc 4.0569 ms/op 4.1492 ms/op 0.98
getCommitteeAssignments - req 1000 vs - 250000 vc 4.3685 ms/op 4.5276 ms/op 0.96
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.3400 ns/op 4.4300 ns/op 0.98
state getBlockRootAtSlot - 250000 vs - 7PWei 952.90 ns/op 858.53 ns/op 1.11
computeProposers - vc 250000 10.382 ms/op 11.134 ms/op 0.93
computeEpochShuffling - vc 250000 102.45 ms/op 103.52 ms/op 0.99
getNextSyncCommittee - vc 250000 168.09 ms/op 179.29 ms/op 0.94

by benchmarkbot/action

@nflaig nflaig force-pushed the nflaig/reuse-stripOffNewlines branch 2 times, most recently from 4a75ba9 to 623b5ab Compare March 19, 2023 09:31
@nflaig nflaig force-pushed the nflaig/reuse-stripOffNewlines branch from 623b5ab to ce714a7 Compare March 19, 2023 09:35
@dapplion dapplion merged commit df1ef97 into unstable Mar 19, 2023
@dapplion dapplion deleted the nflaig/reuse-stripOffNewlines branch March 19, 2023 23:34
twoeths pushed a commit that referenced this pull request Mar 20, 2023
@nflaig @twoeths
Reuse stripOffNewlines to trim keystore password from file (#5283)
a91d502 
* Fix stripOffNewlines to not remove pipe character(s) at the end

* Add unit tests to verify stripOffNewlines works as expected

* Reuse stripOffNewlines to trim keystore password from file
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.6.0 🎉

@wemeetagain
Copy link
Member

🎉 This PR is included in v1.7.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nflaig @wemeetagain @dapplion