Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Upgrade] Go-Ethereum release v1.9.22 #11

Closed

Conversation

baptiste-b-pegasys
Copy link
Owner

TODO

Plan & Analyse

  • Review the Release Notes
  • Review PRs in the section below

As you review, list extra changes and/or tests to be implemented to ensure compatibility with GoQuorum specific features.

Build & Test

  • Pull and checkout PR branch locally, then merge GoQuorum master into this branch
  • Resolve conflicts, taking into account the prior analysis
  • Implement required changes until lint passes
  • Implement required changes until all unit tests pass
  • Implement required changes until acceptance tests pass
  • Implement extra changes and/or tests
  • Verify any left TODOs in the code

Extra Changes & Tests

  • Example title: example description of change/test

Go-Ethereum Release: Noverian Rum (v1.9.22)

  • Version: v1.9.22
  • Published: 2020-09-28T08:24:35Z

Release notes

Geth v1.9.22 is our usual maintenance release, fixing a few bugs and adding some minor features:

  • Add gpo.maxprice CLI flag to override the upper limit of the automatic gas price suggester (#21531).
  • Add BlockNumber method to ethclient to support retrieving the head block's number (#21500).
  • Extend the call tracer to include additional contextual fields for SELFDESTRUCT (#21549, #21564).
  • Unexpose port 8547 from docker images as GraphQL was merged with HTTP RPC (#21556).
  • Support retroactively setting Petersburg if it coincides with Constantinople (#21473).
  • Support gas estimation against arbitrary blocks, not just pending (#21545. #21601).
  • Extend database inspection results with item counters too beside size (#21495).
  • Dynamically move fast pivot point even during chain sync phase (#21529).
  • Start implementing network protocol testers (#21598, #21603, #21604).
  • Support flexible range proofs for snap sync (#21484. #21199, #21250).
  • Extract rlpx into it's own package for easier protocol tests (#21464).
  • Minor API polishes in the Java mobile framework (#21580).
  • Add fuzzer suite for ABI encoding and decoding (#21217).

Bugfixes:

  • Print warning if Whisper is present in the config.toml instead of rejecting (#21544).
  • Handle miner suspends/resumption due to sync more gracefully (#21536, #21547).
  • Fix a light client regression that crashed the node after a sync failure (#21537).

For a full rundown of the changes please consult the Geth 1.9.22 release milestone


As with all our previous releases, you can find the:

Codebase changes assessment

Legend

File Stats: (A) Added, (M) Modified and (R) Removed

Line Stats: (A) Added and (R) Removed

Assessment:

  • ✅ No conflict expected
  • ⚠ Review required to assess changes
  • ‼️ Conflicts expected and review required

39 Pull Requests

🔍 Link Title File Stats
M/A/R
Packages changed
(files changed)
Line Stats
A/R
Top 5 Changed Files
(lines changed)
⚠️ #21535 accounts/abi/bind/backends: reverted some stylistic changes 2/0/0
accounts/abi/bind/backends (2)
25/25
accounts/abi/bind/backends/simulated.go (88)
accounts/abi/bind/backends/simulated_test.go (12)
⚠️ #21537 eth/downloader: only roll back light sync if not fully validating 2/0/0
eth/downloader (2)
12/11
eth/downloader/downloader.go (32)
eth/downloader/downloader_test.go (14)
#21531 cmd, eth: offer maxprice flag for overwritting price cap 5/0/0
cmd/geth (2)
cmd/utils (1)
eth (1)
eth/gasprice (1)
27/3
eth/gasprice/gasprice.go (36)
cmd/utils/flags.go (16)
eth/config.go (4)
cmd/geth/main.go (2)
cmd/geth/usage.go (2)
#21530 core/vm: fix benchmark overflow + prep for precompile repricings 16/0/0
core/vm/testdata/precompiles (15)
core/vm (1)
1007/4
core/vm/testdata/precompiles/blsG1Mul.json (212)
core/vm/testdata/precompiles/blsG1Add.json (208)
core/vm/testdata/precompiles/blsG2Add.json (208)
core/vm/testdata/precompiles/blsG2Mul.json (208)
core/vm/testdata/precompiles/blsG1MultiExp.json (206)
‼️ #21541 go.mod: remove golang.org/x/sync 1/0/0
go.mod (1)
0/1
go.mod (2)
⚠️ #21500 ethclient: add BlockNumber method 1/0/0
ethclient (1)
7/0
ethclient/ethclient.go (14)
⚠️ #21544 cmd/geth: print warning when whisper config is present in toml 1/0/0
cmd/geth (1)
15/0
cmd/geth/config.go (30)
⚠️ #21536 miner: use channels instead of atomics in update loop 2/1/0
miner (3)
196/32
miner/miner_test.go (340)
miner/miner.go (114)
miner/worker.go (2)
#21547 miner: fix regression, add test for starting while download 2/0/0
miner (2)
22/2
miner/miner_test.go (26)
miner/miner.go (22)
#21554 p2p/discover: fix typo in comments 1/0/0
p2p/discover (1)
1/1
p2p/discover/table.go (4)
#21556 dockerfile: remove 8547 port 2/0/0
Dockerfile (1)
Dockerfile.alltools (1)
2/2
Dockerfile (4)
Dockerfile.alltools (4)
#21436 p2p/nodestate: ensure correct callback order 3/0/0
p2p/nodestate (2)
les (1)
317/166
p2p/nodestate/nodestate.go (688)
les/serverpool.go (166)
p2p/nodestate/nodestate_test.go (112)
#21549 Updated call_tracer to be more descriptive 1/0/0
eth/tracers/internal/tracers (1)
8/1
eth/tracers/internal/tracers/call_tracer.js (18)
#21563 rlp: add SplitUint64 2/0/0
rlp (2)
71/0
rlp/raw_test.go (90)
rlp/raw.go (52)
#21236 les, les/lespay/server: refactor client pool 13/9/2
les (11)
les/lespay/server (7)
les/utils (3)
cmd/geth (1)
common/prque (1)
internal/web3ext (1)
3100/1692
les/clientpool.go (1944)
les/lespay/server/balance.go (1218)
les/lespay/server/prioritypool.go (1006)
les/clientpool_test.go (924)
les/lespay/server/balance_test.go (800)
#21564 eth/tracers: regenerate assets 1/0/0
eth/tracers/internal/tracers (1)
3/3
eth/tracers/internal/tracers/assets.go (12)
#21568 COPYING: restore the full text text of GPL 1/0/0
COPYING (1)
57/2
COPYING (118)
#21480 Single point of maintenance for writing and deleting tx lookup indexes 5/0/0
core/rawdb (3)
core (1)
light (1)
31/29
core/rawdb/accessors_indexes.go (76)
core/blockchain.go (16)
core/rawdb/accessors_indexes_test.go (16)
core/rawdb/chain_iterator.go (8)
light/txpool.go (4)
‼️ #21565 ethclient: fix BlockNumber 2/0/0
ethclient (2)
17/1
ethclient/ethclient_test.go (32)
ethclient/ethclient.go (4)
‼️ #21473 Allow setting PetersburgBlock before chain head 2/0/0
params (2)
22/1
params/config_test.go (34)
params/config.go (12)
#21571 les/lespay/server: bump database version 1/0/0
les/lespay/server (1)
6/1
les/lespay/server/clientdb.go (14)
#21217 tests/fuzzers/abi: add fuzzer for fuzzing package accounts/abi 7/2/0
accounts/abi (5)
tests/fuzzers/abi (2)
go.mod (1)
go.sum (1)
266/14
tests/fuzzers/abi/abifuzzer.go (372)
tests/fuzzers/abi/abifuzzer_test.go (100)
accounts/abi/pack.go (46)
go.sum (12)
accounts/abi/unpack.go (10)
⚠️ #21561 cmd/utils: use preconfigured testnet flags instead of networkid 1/0/0
cmd/utils (1)
1/1
cmd/utils/flags.go (4)
#21495 cmd/geth: added counters to the geth inspect report 1/0/0
core/rawdb (1)
114/76
core/rawdb/database.go (380)
#21529 eth/downloader: dynamically move pivot even during chain sync 3/0/0
eth/downloader (3)
192/72
eth/downloader/downloader.go (450)
eth/downloader/testchain_test.go (50)
eth/downloader/downloader_test.go (28)
⚠️ #21439 core: fix a typo in comment 1/0/0
core (1)
1/1
core/block_validator.go (4)
#21540 accounts/abi: improve documentation and names 16/0/0
accounts/abi (11)
accounts/abi/bind (5)
62/61
accounts/abi/bind/bind.go (56)
accounts/abi/argument.go (36)
accounts/abi/unpack.go (34)
accounts/abi/bind/template.go (30)
accounts/abi/abi.go (20)
#21580 mobile: better api for java users 5/0/0
mobile (3)
go.mod (1)
go.sum (1)
93/0
mobile/types.go (88)
go.sum (44)
mobile/geth.go (32)
mobile/common.go (20)
go.mod (2)
#21464 p2p: move rlpx into separate package 5/3/2
p2p (6)
cmd/devp2p (2)
p2p/rlpx (2)
901/742
p2p/rlpx/rlpx.go (1448)
p2p/rlpx/rlpx_test.go (800)
p2p/transport.go (354)
p2p/transport_test.go (296)
cmd/devp2p/rlpxcmd.go (188)
⚠️ #21579 cmd/clef, cmd/geth: use SplitAndTrim from cmd/utils 4/0/0
cmd/utils (2)
cmd/clef (1)
cmd/geth (1)
28/60
cmd/utils/flags.go (86)
cmd/clef/main.go (52)
cmd/geth/retesteth.go (30)
cmd/utils/customflags.go (8)
#21484 trie: fix bad range proof 1/0/0
trie (1)
1/1
trie/proof_test.go (4)
#21199 trie: support empty range proof 2/0/0
trie (2)
50/4
trie/proof_test.go (66)
trie/proof.go (42)
⚠️ #21545 internal/ethapi: add optional parameter blockNrOrHash to estimateGas 1/0/0
internal/ethapi (1)
6/3
internal/ethapi/api.go (18)
#21250 trie: extend range proof 2/0/0
trie (2)
395/182
trie/proof_test.go (712)
trie/proof.go (442)
‼️ #21601 internal/ethapi: fix null pointer exception, add estimate gas to js 2/0/0
internal/ethapi (1)
internal/web3ext (1)
10/0
internal/web3ext/web3ext.go (14)
internal/ethapi/api.go (6)
⚠️ #21598 cmd/devp2p: add eth protocol test suite 8/3/0
eth (4)
cmd/devp2p/internal/ethtest (3)
cmd/devp2p (2)
core/forkid (2)
628/37
cmd/devp2p/internal/ethtest/suite.go (674)
cmd/devp2p/internal/ethtest/types.go (268)
cmd/devp2p/internal/ethtest/chain.go (226)
cmd/devp2p/rlpxcmd.go (102)
core/forkid/forkid.go (30)
#21603 cmd/devp2p/internal/ethtest: update version in handshake 1/0/0
cmd/devp2p/internal/ethtest (1)
1/1
cmd/devp2p/internal/ethtest/suite.go (4)
#21604 cmd/devp2p/internal/ethtest: lower protocol version to 64 1/0/0
cmd/devp2p/internal/ethtest (1)
1/1
cmd/devp2p/internal/ethtest/suite.go (4)
⚠️ #21635 params: update CHTs for Geth v1.9.22 1/0/0
params (1)
12/12
params/config.go (48)

126 Changed files

🔍 File Lines Changed Linked PR
les/clientpool.go 1944 #21236
p2p/rlpx/rlpx.go 1448 #21464
les/lespay/server/balance.go 1218 #21236
les/lespay/server/prioritypool.go 1006 #21236
les/clientpool_test.go 924 #21236
les/lespay/server/balance_test.go 800 #21236
p2p/rlpx/rlpx_test.go 800 #21464
les/balance.go 778 #21236
trie/proof_test.go 770 #21199
#21250
#21484
p2p/nodestate/nodestate.go 688 #21436
cmd/devp2p/internal/ethtest/suite.go 674 #21598
#21604
#21603
les/lespay/server/balance_tracker.go 582 #21236
les/balance_test.go 520 #21236
les/lespay/server/clientdb.go 500 #21236
#21571
⚠️ eth/downloader/downloader.go 482 #21537
#21529
trie/proof.go 468 #21199
#21250
core/rawdb/database.go 380 #21495
tests/fuzzers/abi/abifuzzer.go 372 #21217
miner/miner_test.go 366 #21547
#21536
p2p/transport.go 354 #21464
p2p/transport_test.go 296 #21464
les/lespay/server/clientdb_test.go 288 #21236
cmd/devp2p/internal/ethtest/types.go 268 #21598
les/lespay/server/prioritypool_test.go 258 #21236
cmd/devp2p/internal/ethtest/chain.go 226 #21598
les/api.go 220 #21236
cmd/devp2p/rlpxcmd.go 218 #21598
#21464
core/vm/testdata/precompiles/blsG1Mul.json 212 #21530
core/vm/testdata/precompiles/blsG1Add.json 208 #21530
core/vm/testdata/precompiles/blsG2Add.json 208 #21530
core/vm/testdata/precompiles/blsG2Mul.json 208 #21530
core/vm/testdata/precompiles/blsG2MultiExp.json 206 #21530
core/vm/testdata/precompiles/blsG1MultiExp.json 206 #21530
core/vm/testdata/precompiles/blsPairing.json 200 #21530
core/vm/testdata/precompiles/blsMapG1.json 200 #21530
core/vm/testdata/precompiles/blsMapG2.json 200 #21530
les/serverpool.go 166 #21436
les/utils/timeutils.go 138 #21236
⚠️ miner/miner.go 128 #21547
#21536
COPYING 118 #21568
p2p/nodestate/nodestate_test.go 112 #21436
‼️ p2p/server_test.go 110 #21464
⚠️ cmd/utils/flags.go 106 #21561
#21531
#21579
tests/fuzzers/abi/abifuzzer_test.go 100 #21217
common/prque/lazyqueue.go 98 #21236
les/utils/timeutils_test.go 94 #21236
rlp/raw_test.go 90 #21563
mobile/types.go 88 #21580
‼️ accounts/abi/bind/backends/simulated.go 88 #21535
core/rawdb/accessors_indexes.go 76 #21480
⚠️ params/config.go 60 #21473
#21635
accounts/abi/bind/bind.go 56 #21540
accounts/abi/pack.go 56 #21217
#21540
‼️ go.sum 56 #21217
#21580
rlp/raw.go 52 #21563
‼️ cmd/clef/main.go 52 #21579
les/peer.go 52 #21236
eth/downloader/testchain_test.go 50 #21529
les/utils/expiredvalue.go 46 #21236
les/server.go 44 #21236
accounts/abi/unpack.go 44 #21217
#21540
⚠️ eth/downloader/downloader_test.go 42 #21537
#21529
accounts/abi/argument.go 36 #21540
⚠️ p2p/server.go 36 #21464
core/vm/testdata/precompiles/bn256ScalarMul.json 36 #21530
eth/gasprice/gasprice.go 36 #21531
core/vm/testdata/precompiles/modexp.json 34 #21530
‼️ params/config_test.go 34 #21473
⚠️ les/server_handler.go 34 #21236
core/vm/testdata/precompiles/bn256Add.json 32 #21530
mobile/geth.go 32 #21580
‼️ ethclient/ethclient_test.go 32 #21565
⚠️ cmd/geth/config.go 30 #21544
p2p/peer_test.go 30 #21464
⚠️ accounts/abi/bind/template.go 30 #21540
⚠️ core/forkid/forkid.go 30 #21598
⚠️ cmd/geth/retesteth.go 30 #21579
core/vm/testdata/precompiles/bn256Pairing.json 28 #21530
les/api_test.go 26 #21236
⚠️ internal/ethapi/api.go 24 #21601
#21545
core/vm/contracts_test.go 24 #21530
p2p/message_test.go 22 #21464
accounts/abi/abi.go 20 #21540
mobile/common.go 20 #21580
eth/tracers/internal/tracers/call_tracer.js 18 #21549
‼️ internal/web3ext/web3ext.go 18 #21236
#21601
core/rawdb/accessors_indexes_test.go 16 #21480
⚠️ core/blockchain.go 16 #21480
⚠️ ethclient/ethclient.go 14 #21500
#21565
⚠️ accounts/abi/bind/backend.go 12 #21540
accounts/abi/abi_test.go 12 #21217
#21540
accounts/abi/type.go 12 #21217
#21540
⚠️ accounts/abi/bind/backends/simulated_test.go 12 #21535
eth/tracers/internal/tracers/assets.go 12 #21564
core/vm/testdata/precompiles/ecRecover.json 10 #21530
core/vm/testdata/precompiles/blake2F.json 10 #21530
accounts/abi/topics.go 8 #21540
⚠️ les/test_helper.go 8 #21236
core/rawdb/chain_iterator.go 8 #21480
les/metrics.go 8 #21236
accounts/abi/method.go 8 #21540
‼️ go.mod 8 #21217
#21541
#21580
cmd/utils/customflags.go 8 #21579
‼️ eth/handler.go 6 #21598
⚠️ eth/helper_test.go 6 #21598
eth/discovery.go 6 #21598
cmd/devp2p/main.go 6 #21598
#21464
accounts/abi/event.go 4 #21540
⚠️ eth/protocol_test.go 4 #21598
accounts/abi/event_test.go 4 #21540
accounts/abi/error.go 4 #21217
⚠️ Dockerfile 4 #21556
⚠️ accounts/abi/bind/base.go 4 #21540
⚠️ cmd/geth/les_test.go 4 #21236
⚠️ params/version.go 4
p2p/discover/table.go 4 #21554
⚠️ eth/config.go 4 #21531
⚠️ accounts/abi/bind/auth.go 4 #21540
light/txpool.go 4 #21480
core/forkid/forkid_test.go 4 #21598
Dockerfile.alltools 4 #21556
⚠️ core/block_validator.go 4 #21439
accounts/abi/reflect.go 4 #21540
⚠️ miner/worker.go 2 #21536
⚠️ cmd/geth/usage.go 2 #21531
⚠️ cmd/geth/main.go 2 #21531

karalabe and others added 30 commits September 9, 2020 11:23
eth/downloader: only roll back light sync if not fully validating
* cmd, eth: offer maxprice flag for overwritting price cap

* eth: rename default price cap
…530)

* core/vm/testdata: add gascost expectations to testcases

* core/vm: verify expected gas in tests for precompiles

* core/vm: fix overflow flaw in gas/s calculation
This adds a new client method BlockNumber to fetch the most recent
block number of the chain.
* cmd/geth: print warning when whisper config is present in toml

* Update cmd/geth/config.go

Co-authored-by: Martin Holst Swende <[email protected]>
This PR changes several different things:

- Adds test cases for the miner loop
- Stops the worker if it wasn't already stopped in worker.Close()
- Uses channels instead of atomics in the miner.update() loop

Co-authored-by: Felix Lange <[email protected]>
This PR adds an extra guarantee to NodeStateMachine: it ensures that all
immediate effects of a certain change are processed before any subsequent
effects of any of the immediate effects on the same node. In the original
version, if a cascaded change caused a subscription callback to be called
multiple times for the same node then these calls might have happened in a
wrong chronological order.

For example:

- a subscription to flag0 changes flag1 and flag2
- a subscription to flag1 changes flag3
- a subscription to flag1, flag2 and flag3 was called in the following order:

   [flag1] -> [flag1, flag3]
   [] -> [flag1]
   [flag1, flag3] -> [flag1, flag2, flag3]

This happened because the tree of changes was traversed in a "depth-first
order". Now it is traversed in a "breadth-first order"; each node has a
FIFO queue for pending callbacks and each triggered subscription callback
is added to the end of the list. The already existing guarantees are
retained; no SetState or SetField returns until the callback queue of the
node is empty again. Just like before, it is the responsibility of the
state machine design to ensure that infinite state loops are not possible.
Multiple changes affecting the same node can still happen simultaneously;
in this case the changes can be interleaved in the FIFO of the node but the
correct order is still guaranteed.

A new unit test is also added to verify callback order in the above scenario.
This can be useful when working with raw RLP data.
* les, les/lespay/server: refactor client pool

* les: use ns.Operation and sub calls where needed

* les: fixed tests

* les: removed active/inactive logic from peerSet

* les: removed active/inactive peer logic

* les: fixed linter warnings

* les: fixed more linter errors and added missing metrics

* les: addressed comments

* cmd/geth: fixed TestPriorityClient

* les: simplified clientPool state machine

* les/lespay/server: do not use goroutine for balance callbacks

* internal/web3ext: fix addBalance required parameters

* les: removed freeCapacity, always connect at minCapacity initially

* les: only allow capacity change with priority status

Co-authored-by: rjl493456442 <[email protected]>
When the license was added to the repository, its text was changed (some
sections at the end removed) and, worse, the authors of go-ethereum
tried to claim copyright on the license text.

The correct way to apply GPL to a project is to copy it verbatim.
This change reverts the text of the GPL to the original.
It didn't actually work because it called a method that doesn't
exist. This fixes it also adds a test.

Co-authored-by: Felix Lange <[email protected]>
* Allow setting PetersburgBlock before chainhead

if it is at the same block as ConstantinopleBlock

* Add a negative test
* tests/fuzzers/abi: added abi fuzzer

* accounts/abi: fixed issues found by fuzzing

* tests/fuzzers/abi: update fuzzers, added repro test

* tests/fuzzers/abi: renamed abi_fuzzer to abifuzzer

* tests/fuzzers/abi: updated abi fuzzer

* tests/fuzzers/abi: updated abi fuzzer

* accounts/abi: minor style fix

* go.mod: added go-fuzz dependency

* tests/fuzzers/abi: updated abi fuzzer

* tests/fuzzers/abi: make linter happy

* tests/fuzzers/abi: make linter happy

* tests/fuzzers/abi: comment out false positives
* cmd/utils: use preconfigured testnet flags instead of networkid

* cmd/utils: shorter description

Co-authored-by: Martin Holst Swende <[email protected]>

* Update flags.go

Co-authored-by: Martin Holst Swende <[email protected]>
* database: added counters

* Improved stats for ancient db

* Small improvement

* Better message and added percentage while counting receipts

* Fast counting for receipts

* added info message

* Show both receips itemscount  from ancient db and counted receipts

* Fixed default case

* Removed counter for receipts in ancient store

* Removed counting of receipts present in leveldb
eth/downloader: dynamically move pivot even during chain sync
* accounts: abi/bid/backends; cleaned doc errors, camelCase refactors and anonymous variable assignments

* acounts/abi/bind: doc errors, anonymous parameter assignments

* accounts/abi: doc edits, camelCase refactors

* accounts/abi/bind: review fix

* reverted name changes

* name revert

Co-authored-by: Osoro Bironga <[email protected]>
MariusVanDerWijden and others added 14 commits September 21, 2020 16:33
* (mobile): Adds string representations for types

* mobile: better interfaces add stringer to types

Co-authored-by: sarath <[email protected]>
This change moves the RLPx protocol implementation into a separate package,
p2p/rlpx. The new package can be used to establish RLPx connections for
protocol testing purposes.

Co-authored-by: Felix Lange <[email protected]>
…(#21545)

This allows users to estimate gas on top of arbitrary blocks as well as pending and latest.
Tracing on pending is useful for most users as it takes into account the current txpool while
tracing on latest might be useful for users that have little to know knowledge of the current
transactions in the network.

If blockNrOrHash is not specified, estimateGas defaults to pending
* trie: support non-existent right proof

* trie: improve test

* trie: minor linter fix

Co-authored-by: Péter Szilágyi <[email protected]>
…1601)

* tried to fix

* fix for js api

* fix for nil pointer ex

* rev space

* rev space

* input call formatter
This change adds a test framework for the "eth" protocol and some basic
tests. The tests can be run using the './devp2p rlpx eth-test' command.
params: update CHTs for Geth v1.9.22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
geth upstream upgrade Have some Geth upgrade to process
Projects
None yet
Development

Successfully merging this pull request may close these issues.