Skip to content

Releases: ethereum/tests

Tests update 10.4: winter

18 May 08:50
@winsvega winsvega
v10.4
a380655

Choose a tag to compare

View all tags
Tests update 10.4: winter

The recent tests changes:

Tests development throug out the winter. A few more cases here and there.
Unit tests for typed transactions.

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v10.3

Test format

Tests Fillers

GeneralStateTests/BlockchainTests Filled

Test Regeneration

Update all:

  • Documentation
    PR #1025
    PR #1026

  • RETURNDATACOPY boundaries
    PR #1010

  • Signextend(31, +/- 1) opcode case
    PR #1013

  • CREATE/2 return buffer check
    PR #1015

  • EC Recover bondary tests from TransactionTests
    PR #1018

  • Clarify transaction types validity on different forks
    PR #1023

  • Unit tests for typed transactions
    PR #1031

Upcoming changes:

Implement PoS/Merge tests
Implement malicious rlp tests in StateTest or BlockchainTest format (need more debugging with t8n)
Besu retesteth RPC support (web vmtracing support)
Random state test script

Loading

Tests update 10.3: Test Headers

09 Jan 12:43
@winsvega winsvega
v10.3
005931a

Choose a tag to compare

View all tags
Tests update 10.3: Test Headers

The recent tests changes:

A few more edge test cases around jump, return, revert, maxnonce, maxcodesize, sar/shr/shl.
Updated the test headers to include solidity version.
Transactions include sender field.

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v10.2

Test format

Tests Fillers

"indexes" section can be entirely/partially omitted. the defult value is -1 to all fields.
individual fields in "indexes" can be omitted. the default value is -1.

        "expect" : [
            {
                "indexes" : {  #the entire section can be omitted
                    "data" : -1,
                    "//gas" : -1,  #this field can be omitted.
                    "value" : -1
                },

GeneralStateTests/BlockchainTests Filled

Changes in the "_info" section:
Optional field "labels" with debug info on transaction index labels does not include : label prefix to save space
Added solidity solc version string info

        "_info" : {
            ...
            "labels" : {
                "0" : "transaction1"
            },
            "solidity" : "Version: 0.8.5+commit.a4f2e591.Linux.g++"
            ...
        }

Changes in the "transaction" section:
Added "sender" field to recover the public address from secretKey

        "transaction" : {
            "secretKey" : "0x45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8",
            "sender" : "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b"
            ...
        },
        "transaction" : {
             "r" : "0x8bb6e17e660cdefc2f6f8124e8f11cd74ac6d91ee05d3a1aa6f513335da0e4a7",
             "s" : "0x1d356c9ff34a96d10ec2e0f9d708eea9da2801a37de491c5af43df7c77d56c0c",
             "sender" : "0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b",
             "v" : "0x1b",
            ...
        }

Test Regeneration

Update all:
PR #1009 (regenerate test headers)

  • Documentation
    PR #999
    PR #1005

  • More codesize limit tests around contract creation
    PR #986 (tests for account with max code size)

  • Max account nonce tests
    PR #934 (check that create/create2 with account max nonce does not work)
    PR #989 (delegate call + create with max nonce)

  • Additional tests around return buffer
    PR #994 (check return buffer, revert buffer)

  • Additional test vectors around SAR/SHR/SHL
    PR #1004 (shift operations test vectors)

  • Stress test for max contract code size creation/revert
    PR #992 (create maxCodeSize x 250, OOG/revert, selfdestruct)

  • Correctnes of revert operation (refunds) upon oog/revert inside create/2 and transaction create subcall
    PR #1003 (correctness of revert operation (refunds) in subcalls)

  • Random test generator script for retesteth from ORI
    PR #949 (random test generator script for retesteth)

  • More tests around JUPM/JUMPDEST
    PR #1006

  • Random test failing on besu
    PR #1007

Upcoming changes:

Implement PoS/Merge tests
Implement malicious rlp tests in StateTest or BlockchainTest format (need more debugging with t8n)
Besu retesteth RPC support (web vmtracing support)
Random state test script

Loading

Tests update 10.2: Difficulty Tests

18 Nov 11:46
@winsvega winsvega
v10.2
6401889

Choose a tag to compare

View all tags
Tests update 10.2: Difficulty Tests

The recent tests changes:

Old difficulty tests removed from BasicTests folder
Introduce new difficulty formula test vectors for bomb delay check in DifficultyTests folder

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v10.1

Test format

New test folder for new test suite DifficultyTestsFiller in /src
and DifficultyTests in root

Tests Fillers

GeneralStateTests Filled

Test Regeneration

Update all:
#984

  • Documentation
    PR #981

  • invalid EC signature recovery test
    PR #973

  • remove outdated difficulty tests
    PR #976
    PR ethereum/legacytests#1

  • introduce reworked DifficultyTests
    PR #977

  • difficulty tests for ArrowGlacier
    PR #972

  • more transaction tests
    PR #979
    PR #969
    PR #982
    PR #984
    PR #988

  • Travis validator script for new test formats
    PR #980

Upcoming changes:

Implement malicious rlp tests in StateTest or BlockchainTest format (need more debugging with t8n)
Besu retesteth RPC support (web vmtracing support)
Random state test script

Loading

Tests update 10.1: Transaction Tests

23 Oct 14:11
@winsvega winsvega
v10.1
fbff6fe

Choose a tag to compare

View all tags
Tests update 10.1: Transaction Tests

The recent tests changes:

Legacy tests moved into a separate repository.
Use git submodule update --init to fetch it
Transaction tests reworked to be supported now with geth t9n
Transaction test format changed

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v10.0

Test format

New test folder stEIP3607 in StateTests and BCGeneralStateTests

Tests Fillers

GeneralStateTests Filled

Test Regeneration

Update all:

  • Documentation

  • Blake precompile bug coverage
    PR #948

  • Move legacy tests into a separate repo
    PR #950

  • Transaction from account with code is prohibited
    PR #963

  • Covert old transaction tests to new transaction tests
    with geth t9n transaction test support
    PR #953
    PR #954
    PR #957
    PR #958
    PR #960
    PR #955
    PR #959
    PR #956
    PR #961
    PR #962
    PR #964
    PR #967
    PR #968
    PR #966

Upcoming changes:

==>Difficulty Tests suite for difficulty formula and ice age calculation (just before PoS :lol)
Implement malicious rlp tests in StateTest or BlockchainTest format (need more debugging with t8n)
Besu retesteth RPC support (web vmtracing support)
Random state test script

Loading

Tests update 10.0: Regeneration

15 Sep 19:35
@winsvega winsvega
v10.0
52cb3b3

Choose a tag to compare

View all tags
Tests update 10.0: Regeneration

The recent tests changes:

Regenerate all the tests, change the sorting for pretty looks
Filler hash calculation algorithm is changed
Filled tests _info now has hash to detect changes of the filled test

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.5

Test format

Tests Fillers

GeneralStateTests Filled

_info section now has

"generatedTestHash" : "f3fd299513374b390d8ee5aae37cd3ece229fd069866b01d45e404b700e24064",

A hash field of the filled test.

Test Regeneration

Update all:
PR #931

  • Documentation
    PR #929

  • Test for overlapping memory in a call to precomp 4 (identity)
    PR #938 (Issue #935)

  • The effect of CREATE[2] on the RETURNDATA buffer
    PR #943 (Issue #942)

  • Rework modexp tests
    PR #945

  • Remove archaic difficulty formula tests for test networks
    PR #946

Upcoming changes:

Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Loading

London update 9.0.5: Postfork tests

22 Aug 21:45
@winsvega winsvega
9.0.5
6f41f7b

Choose a tag to compare

View all tags
London update 9.0.5: Postfork tests

The recent tests changes:

A few more tests, fix fillers for test regeneration

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.4

Test format

Tests Fillers

GeneralStateTests Filled

Test Regeneration

Update all:

  • Documentation
    PR #915
    PR #925

  • Remove sender code in LegacyTests
    PR #905
    PR #911
    PR #914

  • More timestamp/difficulty formula tests
    PR #907

  • Prescise OOG on some operations
    Fix issue #909
    PR #913

  • SStore gas costs check
    PR #926

  • Update test fillers in a couple of tests
    PR #927
    PR #930

Upcoming changes:

==> Regenerate all the tests, file hash calculation change
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Loading

London update 9.0.4: Testnet's bugs

27 Jul 09:12
@winsvega winsvega
9.0.4
5d534e3

Choose a tag to compare

View all tags
London update 9.0.4: Testnet's bugs

The recent tests changes:

Test RPC protocol documentation
Tests covering bugs discovered by testnets

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.3

Test format

Tests Fillers

Decimal values can be represented with _ for better readability. Example:

    "balance" : "100_000_000"

GeneralStateTests Filled

Test Regeneration

Update all:

  • Documentation
    PR #894
    PR #904

  • BaseFeePerGas computation bug discovered on besu
    PR #895

  • Intrinsic gas calculation bug in 1559 transaction
    Ropsten reported bug. where value was not included in intrinsic calculation caused an intrinsicly invalid transaction to pass on low baseFee
    PR #898
    PR #901
    PR #903

  • Timestamp difference test (WIP)
    A bit more coverage on timestamp diff/ difficulty formula, more tests to come
    PR #896

  • Split huge test into may small ones for better execution on threads (cosmetic)
    PR #897

  • Fix geth exception name (cosmetic)
    PR #902

Upcoming changes:

London EIP tests
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Loading

London update 9.0.3: Uncle tests

02 Jul 10:46
@winsvega winsvega
9.0.3
e20d7f3

Choose a tag to compare

View all tags
London update 9.0.3: Uncle tests

The recent tests changes:

Template tests around new opcodes and badOpcodes
Uncle tests for London transition

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.2

Test format

GeneralStateTests Fillers

GeneralStateTests Filled

Test Regeneration

Update all:

  • Documentation
    PR #885
    PR #891
    PR #892

  • Remove code from sender account in StateTests
    PR #893

  • Template tests generator and template tests
    Creates tests that run 'X' code inside different scenarious.
    Template tests around basefee selfbalance and badopcodes
    PR #886

  • Uncle tests for London transition
    PR #887
    PR #890

  • Fix baseFee to baseFeePerGas in yml fillers
    PR #884

  • Fix baseFee field
    Cosmetic
    PR #888

Upcoming changes:

London EIP tests
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Loading

London update 9.0.2: Invalid transactions check

12 Jun 20:32
@winsvega winsvega
9.0.2
5d65138

Choose a tag to compare

View all tags
London update 9.0.2: Invalid transactions check

The recent tests changes:

Check transaction rejection rules in Blockchain and StateTests
Rename baseFee to baseFeePerGas
Explicity baseFeePerGas in GeneralStateTests

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.1

Test format

GeneralStateTests Fillers

New field in state tests expect section expectException allows to check transaction rejection reason on different forks

"expect" : [
            {
                "indexes" : {   "data" : -1,   "gas" : 0,   "value" : -1  },
                "network" : [">=Berlin"],
                "expectException" : {
                    ">=Berlin" : "TR_IntrinsicGas"
                },
               ...
]

GeneralStateTests Filled

New field (optional) in post results section expectException indicating that transaction is expected to be rejected. The reason check is optional.

            "Berlin" : [
                {
                    "indexes" : { "data" : 0, "gas" : 0, "value" : 0 },
                    "hash" : "0x4c9c6cf002e6a88a5444662ca9ceb6a116b7b69ced38c470bf6e4a12a6313967",
                    "txbytes" : "0xf862...5",
                    "expectException" : "TR_IntrinsicGas",
                    "logs" : "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347"
                }
            ],

Test Regeneration

Update all:

Upcoming changes:

London EIP tests
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Loading

London update 9.0.1: Fixing issues

05 Jun 22:21
@winsvega winsvega
9.0.1
fa0ab11

Choose a tag to compare

View all tags
London update 9.0.1: Fixing issues

The recent tests changes:

Fixing issues in the tests with EIP1559 formulas
More explicit transaction rejection reason from tools when filling the tests

Announcement

The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/9.0.0

Test format

BlockchainTests Fillers

"invalid" section in transaction replaced with "expectException" similar to the one in BlockHeader overwrite
It is now possible to set expected exception depending on the mining network for a specific transaction
This require test_mineBlocks RPC change in the test protocol and t8ntool response change as well.
The new return on test_mineBlocks is a json

{
   "valid" : true,
   "rejectedTransactions" : [
       {
            "hash" : "0x1122..11",
            "error" ; "Gas Limit reached"
       }
   ]
}

BlockchainTests Filled

If the block test filler had transactions marked "expectException" the filled test would have "transactionSequence" section with transaction order as its imported into state in the current block. if transaction import expected to fail it is marked so with exception. The new section allows clients to check transaction rejection in the blockchain tests. As previously invalid transaction been checked only during the test filling. Invalid transaction do not present in the valid block rlp. Now clients can try importing it on top of the previous block state and make sure the rejection exception is validated.

"transactionSequence" : [
 {
     "rawBytes" : "0xf863802b...4db0d72118",
     "valid" : "false",
     "exception" : "TR_NoFunds"
 },
 {
      "rawBytes" : "0xf863802...b4db0d72118",
     "valid" : "true"
 }
],

Test Regeneration

Update all:

  • Documentation
    PR #867

  • Fixes of pr's errors
    PR #864
    PR #866
    PR #870
    PR #871

  • GasLimit delta checks fix on London
    PR #860

  • EIP1559 transaction gasPrices
    PR #859

  • Update EIP1559 rules
    PR #862

  • Issue 854 Coinbase as hotload account
    #854
    PR #855

  • Correct tests with expected transaction failure reason
    PR #872

  • More transaction exception tests
    PR #873

Upcoming changes:

Transaction rejection check in StateTests
Deprecate "transaction" section in filled GeneralStateTests #828 Rejectd
London EIP tests
Besu retesteth RPC support (web vmtracing support)
Move legacy tests into separate repo (too much space used)

Loading