Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix decrypting keystores with password ending in pipe character #5282

Merged
merged 1 commit into from
Mar 18, 2023

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented Mar 18, 2023

Motivation

Closes #5276

Description

Fix decrypting keystores with password ending in pipe character. This only affects users which provide their password from a file via --importKeystoresPassword and the password ends with a pipe character (|). The pipe character was trimmed off by the regex which is only supposed to remove newline characters and carriage return characters.

Breakdown of our previous regex

  • [...]: Square brackets define a character set. Any character inside the brackets can be matched.
  • \n: Represents a newline character.
  • |: The pipe symbol acts as an OR operator, meaning either the character before or after the pipe can be matched. However, it is unnecessary to include the pipe symbol inside the square brackets, as the square brackets already create a set of characters that can be matched.
  • \r: Represents a carriage return character.
  • +$: The plus sign means "one or more" of the preceding element (in this case, newline or carriage return characters). The dollar sign represents the end of the string.

Updated regex

The new regex /[\n\r]+$/g will only match newline and carriage return characters at the end of the string and replace them with an empty string ""

> "1231321|".replace(/[\n\r]+$/g, "")
'1231321|'
> "1231321\n".replace(/[\n\r]+$/g, "")
'1231321'
> "1231321\r".replace(/[\n\r]+$/g, "")
'1231321'
> "1231321\n\n\r".replace(/[\n\r]+$/g, "")
'1231321'
> "1231321\n\r\n".replace(/[\n\r]+$/g, "")
'1231321'

This only affects users which provide their password from a file via
'--importKeystoresPassword' and the password ends with a pipe character
('|'). The pipe character was trimmed off by the regex which is only
supposed to remove newline characters and carriage return characters.
@nflaig nflaig requested a review from a team as a code owner March 18, 2023 13:03
@github-actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 224bcb8 Previous: 794ebdd Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 901.59 us/op 920.02 us/op 0.98
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 44.259 us/op 43.983 us/op 1.01
BLS verify - blst-native 1.1835 ms/op 1.1813 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 2.4021 ms/op 2.4000 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 5.1539 ms/op 5.1529 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 18.650 ms/op 18.689 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 25.000 us/op 25.038 us/op 1.00
BLS aggregatePubkeys 128 - blst-native 99.145 us/op 97.473 us/op 1.02
getAttestationsForBlock 51.587 ms/op 52.792 ms/op 0.98
isKnown best case - 1 super set check 256.00 ns/op 246.00 ns/op 1.04
isKnown normal case - 2 super set checks 243.00 ns/op 243.00 ns/op 1.00
isKnown worse case - 16 super set checks 248.00 ns/op 243.00 ns/op 1.02
CheckpointStateCache - add get delete 4.9400 us/op 4.7640 us/op 1.04
validate gossip signedAggregateAndProof - struct 2.7011 ms/op 2.6904 ms/op 1.00
validate gossip attestation - struct 1.3039 ms/op 1.2840 ms/op 1.02
pickEth1Vote - no votes 1.2237 ms/op 1.2394 ms/op 0.99
pickEth1Vote - max votes 10.904 ms/op 10.385 ms/op 1.05
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.9466 ms/op 8.6371 ms/op 1.04
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 14.417 ms/op 14.448 ms/op 1.00
pickEth1Vote - Eth1Data fastSerialize value x2048 628.21 us/op 621.25 us/op 1.01
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.7859 ms/op 7.7895 ms/op 1.00
bytes32 toHexString 461.00 ns/op 475.00 ns/op 0.97
bytes32 Buffer.toString(hex) 320.00 ns/op 333.00 ns/op 0.96
bytes32 Buffer.toString(hex) from Uint8Array 507.00 ns/op 532.00 ns/op 0.95
bytes32 Buffer.toString(hex) + 0x 320.00 ns/op 336.00 ns/op 0.95
Object access 1 prop 0.15400 ns/op 0.16500 ns/op 0.93
Map access 1 prop 0.15500 ns/op 0.15600 ns/op 0.99
Object get x1000 6.4980 ns/op 6.4780 ns/op 1.00
Map get x1000 0.60200 ns/op 0.55000 ns/op 1.09
Object set x1000 51.453 ns/op 49.478 ns/op 1.04
Map set x1000 42.776 ns/op 41.584 ns/op 1.03
Return object 10000 times 0.23360 ns/op 0.22220 ns/op 1.05
Throw Error 10000 times 4.1503 us/op 4.0199 us/op 1.03
fastMsgIdFn sha256 / 200 bytes 3.2610 us/op 3.2980 us/op 0.99
fastMsgIdFn h32 xxhash / 200 bytes 260.00 ns/op 268.00 ns/op 0.97
fastMsgIdFn h64 xxhash / 200 bytes 372.00 ns/op 367.00 ns/op 1.01
fastMsgIdFn sha256 / 1000 bytes 10.952 us/op 11.157 us/op 0.98
fastMsgIdFn h32 xxhash / 1000 bytes 376.00 ns/op 381.00 ns/op 0.99
fastMsgIdFn h64 xxhash / 1000 bytes 433.00 ns/op 434.00 ns/op 1.00
fastMsgIdFn sha256 / 10000 bytes 98.595 us/op 99.296 us/op 0.99
fastMsgIdFn h32 xxhash / 10000 bytes 1.8220 us/op 1.8000 us/op 1.01
fastMsgIdFn h64 xxhash / 10000 bytes 1.2870 us/op 1.2820 us/op 1.00
enrSubnets - fastDeserialize 64 bits 1.2330 us/op 1.1980 us/op 1.03
enrSubnets - ssz BitVector 64 bits 453.00 ns/op 462.00 ns/op 0.98
enrSubnets - fastDeserialize 4 bits 159.00 ns/op 157.00 ns/op 1.01
enrSubnets - ssz BitVector 4 bits 462.00 ns/op 463.00 ns/op 1.00
prioritizePeers score -10:0 att 32-0.1 sync 2-0 106.08 us/op 100.96 us/op 1.05
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 139.84 us/op 125.79 us/op 1.11
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 162.04 us/op 160.02 us/op 1.01
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 294.26 us/op 288.22 us/op 1.02
prioritizePeers score 0:0 att 64-1 sync 4-1 352.34 us/op 346.15 us/op 1.02
array of 16000 items push then shift 1.6052 us/op 1.5700 us/op 1.02
LinkedList of 16000 items push then shift 8.6930 ns/op 8.4410 ns/op 1.03
array of 16000 items push then pop 73.254 ns/op 73.032 ns/op 1.00
LinkedList of 16000 items push then pop 8.3570 ns/op 8.1700 ns/op 1.02
array of 24000 items push then shift 2.3340 us/op 2.2654 us/op 1.03
LinkedList of 24000 items push then shift 8.6240 ns/op 8.5420 ns/op 1.01
array of 24000 items push then pop 74.545 ns/op 71.324 ns/op 1.05
LinkedList of 24000 items push then pop 8.1460 ns/op 8.1400 ns/op 1.00
intersect bitArray bitLen 8 12.705 ns/op 12.712 ns/op 1.00
intersect array and set length 8 71.697 ns/op 73.695 ns/op 0.97
intersect bitArray bitLen 128 41.986 ns/op 42.238 ns/op 0.99
intersect array and set length 128 991.65 ns/op 1.0004 us/op 0.99
Buffer.concat 32 items 2.4820 us/op 2.8230 us/op 0.88
Uint8Array.set 32 items 2.9270 us/op 2.4620 us/op 1.19
pass gossip attestations to forkchoice per slot 3.1072 ms/op 3.2273 ms/op 0.96
computeDeltas 3.4636 ms/op 2.8200 ms/op 1.23
computeProposerBoostScoreFromBalances 1.7731 ms/op 1.7720 ms/op 1.00
altair processAttestation - 250000 vs - 7PWei normalcase 2.0591 ms/op 2.1027 ms/op 0.98
altair processAttestation - 250000 vs - 7PWei worstcase 3.2860 ms/op 3.2186 ms/op 1.02
altair processAttestation - setStatus - 1/6 committees join 137.50 us/op 131.95 us/op 1.04
altair processAttestation - setStatus - 1/3 committees join 274.12 us/op 265.50 us/op 1.03
altair processAttestation - setStatus - 1/2 committees join 366.49 us/op 355.83 us/op 1.03
altair processAttestation - setStatus - 2/3 committees join 464.64 us/op 436.74 us/op 1.06
altair processAttestation - setStatus - 4/5 committees join 648.15 us/op 615.97 us/op 1.05
altair processAttestation - setStatus - 100% committees join 753.92 us/op 750.67 us/op 1.00
altair processBlock - 250000 vs - 7PWei normalcase 18.019 ms/op 17.883 ms/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase hashState 29.763 ms/op 25.690 ms/op 1.16
altair processBlock - 250000 vs - 7PWei worstcase 48.075 ms/op 53.130 ms/op 0.90
altair processBlock - 250000 vs - 7PWei worstcase hashState 68.072 ms/op 67.667 ms/op 1.01
phase0 processBlock - 250000 vs - 7PWei normalcase 2.0890 ms/op 1.9500 ms/op 1.07
phase0 processBlock - 250000 vs - 7PWei worstcase 27.791 ms/op 27.304 ms/op 1.02
altair processEth1Data - 250000 vs - 7PWei normalcase 425.79 us/op 468.31 us/op 0.91
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 6.8440 us/op 6.5480 us/op 1.05
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 19.597 us/op 19.443 us/op 1.01
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 8.0660 us/op 8.7950 us/op 0.92
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 6.2420 us/op 6.4040 us/op 0.97
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 74.682 us/op 74.047 us/op 1.01
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 590.85 us/op 615.32 us/op 0.96
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 908.20 us/op 890.88 us/op 1.02
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 897.97 us/op 840.10 us/op 1.07
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.1924 ms/op 2.2379 ms/op 0.98
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.6651 ms/op 1.4321 ms/op 1.16
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.8776 ms/op 3.5434 ms/op 1.09
Tree 40 250000 create 309.63 ms/op 292.93 ms/op 1.06
Tree 40 250000 get(125000) 177.44 ns/op 170.41 ns/op 1.04
Tree 40 250000 set(125000) 869.51 ns/op 895.16 ns/op 0.97
Tree 40 250000 toArray() 16.841 ms/op 17.981 ms/op 0.94
Tree 40 250000 iterate all - toArray() + loop 16.214 ms/op 16.481 ms/op 0.98
Tree 40 250000 iterate all - get(i) 64.408 ms/op 63.746 ms/op 1.01
MutableVector 250000 create 9.4302 ms/op 9.6029 ms/op 0.98
MutableVector 250000 get(125000) 6.2270 ns/op 6.3050 ns/op 0.99
MutableVector 250000 set(125000) 253.46 ns/op 247.35 ns/op 1.02
MutableVector 250000 toArray() 2.6757 ms/op 2.6601 ms/op 1.01
MutableVector 250000 iterate all - toArray() + loop 2.7932 ms/op 2.7591 ms/op 1.01
MutableVector 250000 iterate all - get(i) 1.5018 ms/op 1.5068 ms/op 1.00
Array 250000 create 2.7450 ms/op 2.4643 ms/op 1.11
Array 250000 clone - spread 1.0836 ms/op 1.1747 ms/op 0.92
Array 250000 get(125000) 0.51800 ns/op 0.56300 ns/op 0.92
Array 250000 set(125000) 0.59900 ns/op 0.64900 ns/op 0.92
Array 250000 iterate all - loop 89.719 us/op 81.773 us/op 1.10
effectiveBalanceIncrements clone Uint8Array 300000 22.310 us/op 25.376 us/op 0.88
effectiveBalanceIncrements clone MutableVector 300000 319.00 ns/op 374.00 ns/op 0.85
effectiveBalanceIncrements rw all Uint8Array 300000 165.12 us/op 166.52 us/op 0.99
effectiveBalanceIncrements rw all MutableVector 300000 76.080 ms/op 80.637 ms/op 0.94
phase0 afterProcessEpoch - 250000 vs - 7PWei 111.36 ms/op 112.15 ms/op 0.99
phase0 beforeProcessEpoch - 250000 vs - 7PWei 41.563 ms/op 32.076 ms/op 1.30
altair processEpoch - mainnet_e81889 319.49 ms/op 324.54 ms/op 0.98
mainnet_e81889 - altair beforeProcessEpoch 62.741 ms/op 66.003 ms/op 0.95
mainnet_e81889 - altair processJustificationAndFinalization 17.233 us/op 16.520 us/op 1.04
mainnet_e81889 - altair processInactivityUpdates 5.4097 ms/op 5.4659 ms/op 0.99
mainnet_e81889 - altair processRewardsAndPenalties 66.959 ms/op 66.981 ms/op 1.00
mainnet_e81889 - altair processRegistryUpdates 2.9370 us/op 2.9440 us/op 1.00
mainnet_e81889 - altair processSlashings 454.00 ns/op 471.00 ns/op 0.96
mainnet_e81889 - altair processEth1DataReset 480.00 ns/op 517.00 ns/op 0.93
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2307 ms/op 1.2900 ms/op 0.95
mainnet_e81889 - altair processSlashingsReset 4.9880 us/op 5.3420 us/op 0.93
mainnet_e81889 - altair processRandaoMixesReset 4.4630 us/op 4.6310 us/op 0.96
mainnet_e81889 - altair processHistoricalRootsUpdate 560.00 ns/op 575.00 ns/op 0.97
mainnet_e81889 - altair processParticipationFlagUpdates 2.3600 us/op 2.4680 us/op 0.96
mainnet_e81889 - altair processSyncCommitteeUpdates 485.00 ns/op 436.00 ns/op 1.11
mainnet_e81889 - altair afterProcessEpoch 114.52 ms/op 116.88 ms/op 0.98
phase0 processEpoch - mainnet_e58758 315.26 ms/op 315.65 ms/op 1.00
mainnet_e58758 - phase0 beforeProcessEpoch 126.15 ms/op 123.40 ms/op 1.02
mainnet_e58758 - phase0 processJustificationAndFinalization 15.447 us/op 16.816 us/op 0.92
mainnet_e58758 - phase0 processRewardsAndPenalties 55.914 ms/op 54.825 ms/op 1.02
mainnet_e58758 - phase0 processRegistryUpdates 7.6730 us/op 7.6120 us/op 1.01
mainnet_e58758 - phase0 processSlashings 493.00 ns/op 502.00 ns/op 0.98
mainnet_e58758 - phase0 processEth1DataReset 495.00 ns/op 530.00 ns/op 0.93
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 939.26 us/op 940.94 us/op 1.00
mainnet_e58758 - phase0 processSlashingsReset 2.9990 us/op 3.0350 us/op 0.99
mainnet_e58758 - phase0 processRandaoMixesReset 4.6260 us/op 4.5720 us/op 1.01
mainnet_e58758 - phase0 processHistoricalRootsUpdate 574.00 ns/op 636.00 ns/op 0.90
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.9960 us/op 4.0080 us/op 1.00
mainnet_e58758 - phase0 afterProcessEpoch 94.054 ms/op 95.245 ms/op 0.99
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.1850 ms/op 1.2194 ms/op 0.97
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.3889 ms/op 1.4341 ms/op 0.97
altair processInactivityUpdates - 250000 normalcase 19.810 ms/op 19.387 ms/op 1.02
altair processInactivityUpdates - 250000 worstcase 23.660 ms/op 21.774 ms/op 1.09
phase0 processRegistryUpdates - 250000 normalcase 6.3340 us/op 6.2530 us/op 1.01
phase0 processRegistryUpdates - 250000 badcase_full_deposits 237.31 us/op 240.55 us/op 0.99
phase0 processRegistryUpdates - 250000 worstcase 0.5 113.31 ms/op 123.41 ms/op 0.92
altair processRewardsAndPenalties - 250000 normalcase 56.514 ms/op 56.023 ms/op 1.01
altair processRewardsAndPenalties - 250000 worstcase 56.154 ms/op 48.172 ms/op 1.17
phase0 getAttestationDeltas - 250000 normalcase 6.2570 ms/op 6.4186 ms/op 0.97
phase0 getAttestationDeltas - 250000 worstcase 6.4944 ms/op 6.1917 ms/op 1.05
phase0 processSlashings - 250000 worstcase 3.4337 ms/op 3.5208 ms/op 0.98
altair processSyncCommitteeUpdates - 250000 172.16 ms/op 171.86 ms/op 1.00
BeaconState.hashTreeRoot - No change 253.00 ns/op 250.00 ns/op 1.01
BeaconState.hashTreeRoot - 1 full validator 50.656 us/op 49.511 us/op 1.02
BeaconState.hashTreeRoot - 32 full validator 503.76 us/op 521.01 us/op 0.97
BeaconState.hashTreeRoot - 512 full validator 5.5177 ms/op 5.0928 ms/op 1.08
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 60.730 us/op 61.569 us/op 0.99
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 862.43 us/op 880.35 us/op 0.98
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.036 ms/op 10.977 ms/op 1.01
BeaconState.hashTreeRoot - 1 balances 47.184 us/op 48.765 us/op 0.97
BeaconState.hashTreeRoot - 32 balances 449.47 us/op 434.50 us/op 1.03
BeaconState.hashTreeRoot - 512 balances 4.3006 ms/op 4.2930 ms/op 1.00
BeaconState.hashTreeRoot - 250000 balances 74.404 ms/op 73.279 ms/op 1.02
aggregationBits - 2048 els - zipIndexesInBitList 16.592 us/op 16.369 us/op 1.01
regular array get 100000 times 32.574 us/op 32.002 us/op 1.02
wrappedArray get 100000 times 32.635 us/op 31.999 us/op 1.02
arrayWithProxy get 100000 times 16.205 ms/op 15.593 ms/op 1.04
ssz.Root.equals 544.00 ns/op 540.00 ns/op 1.01
byteArrayEquals 534.00 ns/op 519.00 ns/op 1.03
shuffle list - 16384 els 6.7259 ms/op 6.7623 ms/op 0.99
shuffle list - 250000 els 99.091 ms/op 99.430 ms/op 1.00
processSlot - 1 slots 8.4880 us/op 8.5430 us/op 0.99
processSlot - 32 slots 1.3212 ms/op 1.3293 ms/op 0.99
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 185.60 us/op 190.50 us/op 0.97
getCommitteeAssignments - req 1 vs - 250000 vc 2.8443 ms/op 2.8952 ms/op 0.98
getCommitteeAssignments - req 100 vs - 250000 vc 3.9827 ms/op 4.0754 ms/op 0.98
getCommitteeAssignments - req 1000 vs - 250000 vc 4.2640 ms/op 4.4498 ms/op 0.96
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.2700 ns/op 4.3500 ns/op 0.98
state getBlockRootAtSlot - 250000 vs - 7PWei 817.59 ns/op 921.74 ns/op 0.89
computeProposers - vc 250000 10.091 ms/op 10.738 ms/op 0.94
computeEpochShuffling - vc 250000 98.584 ms/op 102.04 ms/op 0.97
getNextSyncCommittee - vc 250000 176.98 ms/op 176.75 ms/op 1.00

by benchmarkbot/action

@nflaig nflaig merged commit 9f7b830 into unstable Mar 18, 2023
@nflaig nflaig deleted the nflaig/fix-keystore-pwfile-decrypt branch March 18, 2023 13:24
@philknows philknows added this to the v1.6.0 milestone Mar 18, 2023
twoeths pushed a commit that referenced this pull request Mar 20, 2023
This only affected users which provide their password from a file via
'--importKeystoresPassword' and the password ends with a pipe character
('|'). The pipe character was trimmed off by the regex which is only
supposed to remove newline characters and carriage return characters.
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.6.0 🎉

@wemeetagain
Copy link
Member

🎉 This PR is included in v1.7.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

VC cannot decrypt keystores with encryption password ending in specific character(s)
4 participants