Skip to content

Retesteth v0.3.0 - Shanghai
Compare
Choose a tag to compare
@winsvega winsvega released this 20 Mar 12:35
· 244 commits to master since this release
v0.3.0-shanghai
e98b5e6
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Many changes: proto support for pyspec, new clients, Shanghai logic
Introducing more debug options (--statediff, --poststate, --vmtrace) for intermediate states/transactions in blockchain tests
Net tests suites: -t EIPTEsts, -t EOFTests
Compatible test release: https://github.com/ethereum/tests/releases/tag/v12

New options:

    retesteth file.json                  Run/Fill the test directly (--singletest detected if in tests path, --testfile if outside)
    retesteth <Folder>                   Run/Fill the test directory (if inside testpath detect -t suite, not all suites supported)
    --exportcall <Folder>                Option to export t8ntool interactions
    --vmtrace <Folder>                   Option to export transaction traces into files
    --vmtrace x:y                        Trace block x, transaction y

    --vmtraceraw <Folder>                Option to export transaction traces into files
    --vmtraceraw x:y                     Trace block x, transaction y

      (intermidiate state for individual transaction in bc tests requires --filltests command)
    --poststate  <Folder>                Option to export transaction states into files (when running state tests)
    --poststate x:y                      Show poststate block x, transaction y
    --statediff                          Print statediff post vs pre in StateTests
    --statediff x:ytox2:y2               Show statediff from block x, transaction y  to block x2, transaction y2
    --statediff xtoy                     Show statediff from block x  to block y

    --chainid  id                        To override transaction chain id when generating or running the test

    --verbosity <channel>                Set logs channels. 'STATS|RPC|TESTLOG|LOWLOG|SOCKET|STATE'
    --getvectors                         Output all subunits of the given test (disables execution)
    --filloutdated                       Run only unfilled test fillers

Removed option:

    --testfolder                   No longer required as new test folders automatically registrate within boost

Announcment

Retesteth is ethereum test runner / generator tool for the tests (https://github.com/ethereum/tests)
Uses existing clients to run/generate the tests. See documentation. (https://ethereum-tests.readthedocs.io/en/latest/)
The test process diagram: http://tinyurl.com/2ctak8yn

Testing stats

Testing results are available at http://retesteth.ethdevops.io/
There is a web tool for vmtracing the tests using supported clients and retesteth: http://retesteth.ethdevops.io/web/
All blockchain tests are being run by hive tool: https://hivetests.ethdevops.io/

Changelog

Interesting

  • any new folder in supported test src suites will be automatically registered into boost
  • new unregistered folders and subfolders (1 lvl depth) automatically register within boost
  • custom compiler shell script in client configs to define parser for src code in test fillers (example, docs)
  • cmake configure option -DLOCALDEPS to be able to use locally installed dependencies and don't use hunter (BOOST)
  • intermidiate poststate and statediff debug in blockchain tests
  • vmtraceraw "outfolder" option to export transaction traces into files
  • poststate "outfolder" option to export transaction states into files (when running state tests)
  • verbosity better selection of logging output
  • getvectors option to print all test vectors commands in particular test suite
  • filloutdated option to refill only those tests that hash is different to the filler file
  • t8ntool calculate baseFee support
  • forceupdate option allow to fill LegacyTests
  • singlenet option in test run mode does not throw an Error
  • chainID defined for individual fork in forks config files
  • web utility scripts added to the repo
  • statediff option print state change in state tests
  • fork+xxxx automatically asks t8n clients to run on fork+xxxx (where xxxx is eip number) rules
  • support Shanghai logic (adds withdrawalsRoot and withdrawals fields to the bc tests)

Client Configs

  • integrate ethereumjs config
  • integrate etc (ethereum classic) config
  • integrate nimbus config
  • integrate evmone config (develop)
  • switch besu from rpc config to t8n
  • new option checkDifficulty to verify ETH difficulty calculation of the t8ntool client
  • new option checkBasefee to verify ETH baseFee calculation of the t8ntool client
  • new option checkBasefee to verify ETH baseFee calculation of the t8ntool client
  • new array fillerSkipForks to skip filling tests on forks (forks + eips) that are not supported
  • new option calculateBasefee to calculate basefee for the t8ntool client that does not support it
  • new option calculateDifficulty to force retesteth calculated difficulty (in case client does not support it)
  • new option support1559 to disable blockheader auto convertion to Merge/1559 rules (for etc clients)
  • new option transactionAsJson to send t8ntool tx files in json form not RLP (for evmone type clients)
  • integrate pyspec .py fillers support (experimental)
  • update default yul.sh script
  • update default start.sh for t8ntool clients (include --erorlog option parsing and better exception check)
  • update dretesteth.sh

Technical

  • print retesteth config path
  • update circle config builds, support mac builds
  • refactor compiler.cpp, main.cpp, options.cpp, statetests, testsuite runner and more
  • update to CXX17
  • remove libff MPIR scrypt AES dependencies
  • remove unused boost_program_options
  • update hunter gate to https://github.com/cpp-pm/hunter/archive/v0.24.7.tar.gz
  • refactor includes in .h and .cpp files. remove redundancies
  • fix fs=boost::filesystem in headers compiler error
  • make dataobject a separate lib
  • nodes option autoresolve hostname (before only accepting IP)
  • hardcode new test folders, fix --testfolder option
  • testfile have now the same logic as normal test execution/generation
  • refactor test versioning (test hash verification, test filler verification, test without fillers)
  • refactor Blockchain test run/generation functions
  • better check of error log from t8ntool
  • disable RPC support for GeneralStateTests generation (do not ask to calculate genesis)
  • don't stop on test's _info section validaton errors

ubuntu 18.04.3:
http://retesteth.ethdevops.io/release/0.3.0-shanghai/ubuntu-18.04.3/retesteth-0.3.0-shanghai-ubuntu-18.04.3

Might need the dependecies install (but better to be run on Ubuntu20):

- sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y
- sudo apt-get update
- sudo apt-get install gcc-9
- sudo apt-get install --only-upgrade libstdc++6

macOS-XCode12:
http://retesteth.ethdevops.io/release/0.3.0-shanghai/macOS-XCode12/retesteth-0.3.0-shanghai-macOS-XCode12

macOS-XCode14:
http://retesteth.ethdevops.io/release/0.3.0-shanghai/macOS-XCode14/retesteth-0.3.0-shanghai-macOS-XCode14

Docker:
(!Better version of docker is comming with other clients support!)
geth version: geth version 1.11.5-unstable-5d23d21f-20230320 Git Commit (around): 5d23d21f
geth evm version: geth version 1.11.5-unstable-5d23d21f
lllc version: Version: 0.5.14-develop.2022.7.30+commit.a096d7a9.Linux.g++
solidity version: Version: 0.8.17+commit.8df45f5f.Linux.g++
http://retesteth.ethdevops.io/release/0.3.0-shanghai/dretesteth-0.3.0-shanghai.tar

Documentation:
https://ethereum-tests.readthedocs.io/en/latest/

Assets 2
Loading