|
2 | 2 |
|
3 | 3 | Depending on the changes introduced by an EIP, the following template is the minimum baseline to guarantee test coverage of the Execution Layer features.
|
4 | 4 |
|
5 |
| -## General |
| 5 | +## <!-- id:general --> General |
6 | 6 |
|
7 |
| -- [ ] Code coverage |
8 |
| - - [ ] Run produced tests against [EELS](https://github.com/ethereum/execution-specs) and verify that line code coverage of new added lines for the EIP is 100%, with only exceptions being unreachable code lines. |
9 |
| - - [ ] Run coverage on the test code itself (as a basic logic sanity check), i.e., `uv run fill --cov tests`. |
10 |
| - - [ ] Optional - Run against a second client and verify sufficient code coverage over new code added for the EIP. |
11 |
| -- [ ] Fuzzing |
12 |
| - - [ ] TBD |
| 7 | +- [ ] <!-- id:general/code_coverage --> Code coverage |
| 8 | + - [ ] <!-- id:general/code_coverage/eels --> Run produced tests against [EELS](https://github.com/ethereum/execution-specs) and verify that line code coverage of new added lines for the EIP is 100%, with only exceptions being unreachable code lines. |
| 9 | + - [ ] <!-- id:general/code_coverage/test_coverage --> Run coverage on the test code itself (as a basic logic sanity check), i.e., `uv run fill --cov tests`. |
| 10 | + - [ ] <!-- id:general/code_coverage/second_client --> Optional - Run against a second client and verify sufficient code coverage over new code added for the EIP. |
| 11 | +- [ ] <!-- id:general/fuzzing --> Fuzzing |
13 | 12 |
|
14 | 13 | ## <!-- id:new_opcode --> New Opcode
|
15 | 14 |
|
@@ -363,85 +362,85 @@ The EIP removes one or more precompiles from the existing list of precompiles.
|
363 | 362 |
|
364 | 363 | ## <!-- id:new_block_body_field --> New Block Body Field
|
365 | 364 |
|
366 |
| -### Test Vectors |
| 365 | +### <!-- id:new_block_body_field/test --> Test Vectors |
367 | 366 |
|
368 |
| -- [ ] Value behavior |
369 |
| - - [ ] Verify, given multiple initial values, that a block is accepted if the value is the correct expected for the current block, depending on the circumstances that affect the value as defined in the EIP. |
370 |
| - - [ ] Verify, given multiple initial values, that a block is rejected if the value is modified (using appropriate `block`) to an incorrect value for the current block, depending on the circumstances that affect the value as defined in the EIP. |
371 |
| -- [ ] Fork transition |
372 |
| - - [ ] Verify that a block containing the new block body field before the activation of the fork is invalid. |
373 |
| - - [ ] Verify that a block lacking the new block field at the activation of the fork is invalid. |
| 367 | +- [ ] <!-- id:new_block_body_field/test/value_behavior --> Value behavior |
| 368 | + - [ ] <!-- id:new_block_body_field/test/value_behavior/accept --> Verify, given multiple initial values, that a block is accepted if the value is the correct expected for the current block, depending on the circumstances that affect the value as defined in the EIP. |
| 369 | + - [ ] <!-- id:new_block_body_field/test/value_behavior/reject --> Verify, given multiple initial values, that a block is rejected if the value is modified (using appropriate `block`) to an incorrect value for the current block, depending on the circumstances that affect the value as defined in the EIP. |
| 370 | +- [ ] <!-- id:new_block_body_field/test/fork_transition --> Fork transition |
| 371 | + - [ ] <!-- id:new_block_body_field/test/fork_transition/before --> Verify that a block containing the new block body field before the activation of the fork is invalid. |
| 372 | + - [ ] <!-- id:new_block_body_field/test/fork_transition/after --> Verify that a block lacking the new block field at the activation of the fork is invalid. |
374 | 373 |
|
375 |
| -### Framework Changes |
| 374 | +### <!-- id:new_block_body_field/framework --> Framework Changes |
376 | 375 |
|
377 |
| -- [ ] Value behavior |
378 |
| - - [ ] Verify, given multiple initial values, that a block is accepted if the value is correctly modified for the current block, depending on the circumstances that affect the value as defined in the EIP. |
379 |
| - - [ ] Verify, given multiple initial values, that a block is rejected if the value is incorrectly modified for the current block, depending on the circumstances that affect the value as defined in the EIP. |
380 |
| -- [ ] Add the new body field to the relevant objects: |
381 |
| - - [ ] `ethereum_test_fixtures.FixtureBlockBase` |
382 |
| - - [ ] `ethereum_test_fixtures.FixtureEngineNewPayload` |
383 |
| - - [ ] `ethereum_test_specs.Block` |
384 |
| -- [ ] Modify `ethereum_test_specs.BlockchainTest` filling behavior to account for the new block field. |
| 376 | +- [ ] <!-- id:new_block_body_field/framework/value_behavior --> Value behavior |
| 377 | + - [ ] <!-- id:new_block_body_field/framework/value_behavior/accept --> Verify, given multiple initial values, that a block is accepted if the value is correctly modified for the current block, depending on the circumstances that affect the value as defined in the EIP. |
| 378 | + - [ ] <!-- id:new_block_body_field/framework/value_behavior/reject --> Verify, given multiple initial values, that a block is rejected if the value is incorrectly modified for the current block, depending on the circumstances that affect the value as defined in the EIP. |
| 379 | +- [ ] <!-- id:new_block_body_field/framework/objects --> Add the new body field to the relevant objects: |
| 380 | + - [ ] <!-- id:new_block_body_field/framework/objects/fixture_block --> `ethereum_test_fixtures.FixtureBlockBase` |
| 381 | + - [ ] <!-- id:new_block_body_field/framework/objects/fixture_engine --> `ethereum_test_fixtures.FixtureEngineNewPayload` |
| 382 | + - [ ] <!-- id:new_block_body_field/framework/objects/block --> `ethereum_test_specs.Block` |
| 383 | +- [ ] <!-- id:new_block_body_field/framework/filling --> Modify `ethereum_test_specs.BlockchainTest` filling behavior to account for the new block field. |
385 | 384 |
|
386 | 385 | ## <!-- id:gas_cost_changes --> Gas Cost Changes
|
387 | 386 |
|
388 |
| -### Test Vectors |
| 387 | +### <!-- id:gas_cost_changes/test --> Test Vectors |
389 | 388 |
|
390 |
| -- [ ] Gas Usage: Measure and store the gas usage during the operations affected by the gas cost changes and verify the updated behavior. |
391 |
| -- [ ] Out-of-gas: Verify the operations affected by the gas cost changes can run out-of-gas with the updated limits. |
392 |
| -- [ ] Fork transition: Verify gas costs are: |
393 |
| - - [ ] Unaffected before the fork activation block. |
394 |
| - - [ ] Updated on and after fork activation block. |
| 389 | +- [ ] <!-- id:gas_cost_changes/test/gas_usage --> Gas Usage: Measure and store the gas usage during the operations affected by the gas cost changes and verify the updated behavior. |
| 390 | +- [ ] <!-- id:gas_cost_changes/test/out_of_gas --> Out-of-gas: Verify the operations affected by the gas cost changes can run out-of-gas with the updated limits. |
| 391 | +- [ ] <!-- id:gas_cost_changes/test/fork_transition --> Fork transition: Verify gas costs are: |
| 392 | + - [ ] <!-- id:gas_cost_changes/test/fork_transition/before --> Unaffected before the fork activation block. |
| 393 | + - [ ] <!-- id:gas_cost_changes/test/fork_transition/after --> Updated on and after fork activation block. |
395 | 394 |
|
396 |
| -### Framework Changes |
| 395 | +### <!-- id:gas_cost_changes/framework --> Framework Changes |
397 | 396 |
|
398 |
| -- [ ] Modify `transaction_intrinsic_cost_calculator` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` if the EIP affects intrinsic gas cost calculation. |
399 |
| -- [ ] Modify `transaction_data_floor_cost_calculator` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` if the EIP affects calldata floor cost. |
400 |
| -- [ ] Modify `memory_expansion_gas_calculator` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` if the EIP affects memory expansion gas cost calculation. |
401 |
| -- [ ] Modify `gas_costs` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` if the EIP affects specific opcode gas costs. |
| 397 | +- [ ] <!-- id:gas_cost_changes/framework/intrinsic_cost --> Modify `transaction_intrinsic_cost_calculator` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` if the EIP affects intrinsic gas cost calculation. |
| 398 | +- [ ] <!-- id:gas_cost_changes/framework/data_floor --> Modify `transaction_data_floor_cost_calculator` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` if the EIP affects calldata floor cost. |
| 399 | +- [ ] <!-- id:gas_cost_changes/framework/memory_expansion --> Modify `memory_expansion_gas_calculator` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` if the EIP affects memory expansion gas cost calculation. |
| 400 | +- [ ] <!-- id:gas_cost_changes/framework/opcode_costs --> Modify `gas_costs` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` if the EIP affects specific opcode gas costs. |
402 | 401 |
|
403 | 402 | ## <!-- id:gas_refunds_changes --> Gas Refunds Changes
|
404 | 403 |
|
405 |
| -### Test Vectors |
406 |
| - |
407 |
| -- [ ] Refund calculation: Verify that the refund does not exceed `gas_used // MAX_REFUND_QUOTIENT` (`MAX_REFUND_QUOTIENT==5` in [EIP-3529](https://eips.ethereum.org/EIPS/eip-3529)) in the following scenarios: |
408 |
| - - [ ] `refund == gas_used // MAX_REFUND_QUOTIENT + 1` |
409 |
| - - [ ] `refund == gas_used // MAX_REFUND_QUOTIENT` |
410 |
| - - [ ] `refund == gas_used // MAX_REFUND_QUOTIENT - 1` |
411 |
| -- [ ] Exceptional Abort: |
412 |
| - - [ ] If the operation causing the refund can be reverted, verify the refund is not applied if the following cases: |
413 |
| - - [ ] `REVERT` |
414 |
| - - [ ] Out-of-gas |
415 |
| - - [ ] Invalid opcode |
416 |
| - - [ ] `REVERT` of an upper call frame |
417 |
| - - [ ] If the operation causing the refund cannot be reverted (e.g. in the case of a transaction-scoped operation such as authorization refunds in EIP-7702), verify the refund is still applied even in the following cases: |
418 |
| - - [ ] `REVERT` at the top call frame |
419 |
| - - [ ] Out-of-gas at the top call frame |
420 |
| - - [ ] Invalid opcode at the top call frame |
421 |
| -- [ ] Cross-Functional Test: Verify the following tests are updated to support the new type of refunds: |
422 |
| - - [ ] `tests/prague/eip7623_increase_calldata_cost/test_refunds.py` |
423 |
| - |
424 |
| -### Framework Changes |
| 404 | +### <!-- id:gas_refunds_changes/test --> Test Vectors |
| 405 | + |
| 406 | +- [ ] <!-- id:gas_refunds_changes/test/refund_calculation --> Refund calculation: Verify that the refund does not exceed `gas_used // MAX_REFUND_QUOTIENT` (`MAX_REFUND_QUOTIENT==5` in [EIP-3529](https://eips.ethereum.org/EIPS/eip-3529)) in the following scenarios: |
| 407 | + - [ ] <!-- id:gas_refunds_changes/test/refund_calculation/over --> `refund == gas_used // MAX_REFUND_QUOTIENT + 1` |
| 408 | + - [ ] <!-- id:gas_refunds_changes/test/refund_calculation/exact --> `refund == gas_used // MAX_REFUND_QUOTIENT` |
| 409 | + - [ ] <!-- id:gas_refunds_changes/test/refund_calculation/under --> `refund == gas_used // MAX_REFUND_QUOTIENT - 1` |
| 410 | +- [ ] <!-- id:gas_refunds_changes/test/exceptional_abort --> Exceptional Abort: |
| 411 | + - [ ] <!-- id:gas_refunds_changes/test/exceptional_abort/revertable --> If the operation causing the refund can be reverted, verify the refund is not applied if the following cases: |
| 412 | + - [ ] <!-- id:gas_refunds_changes/test/exceptional_abort/revertable/revert --> `REVERT` |
| 413 | + - [ ] <!-- id:gas_refunds_changes/test/exceptional_abort/revertable/out_of_gas --> Out-of-gas |
| 414 | + - [ ] <!-- id:gas_refunds_changes/test/exceptional_abort/revertable/invalid_opcode --> Invalid opcode |
| 415 | + - [ ] <!-- id:gas_refunds_changes/test/exceptional_abort/revertable/upper_revert --> `REVERT` of an upper call frame |
| 416 | + - [ ] <!-- id:gas_refunds_changes/test/exceptional_abort/non_revertable --> If the operation causing the refund cannot be reverted (e.g. in the case of a transaction-scoped operation such as authorization refunds in EIP-7702), verify the refund is still applied even in the following cases: |
| 417 | + - [ ] <!-- id:gas_refunds_changes/test/exceptional_abort/non_revertable/revert --> `REVERT` at the top call frame |
| 418 | + - [ ] <!-- id:gas_refunds_changes/test/exceptional_abort/non_revertable/out_of_gas --> Out-of-gas at the top call frame |
| 419 | + - [ ] <!-- id:gas_refunds_changes/test/exceptional_abort/non_revertable/invalid_opcode --> Invalid opcode at the top call frame |
| 420 | +- [ ] <!-- id:gas_refunds_changes/test/cross_functional --> Cross-Functional Test: Verify the following tests are updated to support the new type of refunds: |
| 421 | + - [ ] <!-- id:gas_refunds_changes/test/cross_functional/calldata_cost --> `tests/prague/eip7623_increase_calldata_cost/test_refunds.py` |
| 422 | + |
| 423 | +### <!-- id:gas_refunds_changes/framework --> Framework Changes |
425 | 424 |
|
426 | 425 | N/A
|
427 | 426 |
|
428 | 427 | ## <!-- id:blob_count_changes --> Blob Count Changes
|
429 | 428 |
|
430 |
| -### <!-- id:blob_count_changes_test --> Test Vectors |
| 429 | +### <!-- id:blob_count_changes/test --> Test Vectors |
431 | 430 |
|
432 |
| -- [ ] Verify tests in `tests/cancun/eip4844_blobs` were correctly and automatically updated to take into account the new blob count values at the new fork activation block. |
| 431 | +- [ ] <!-- id:blob_count_changes/test/update_eip4844_blobs --> Verify tests in `tests/cancun/eip4844_blobs` were correctly and automatically updated to take into account the new blob count values at the new fork activation block. |
433 | 432 |
|
434 |
| -### <!-- id:blob_count_changes_framework --> Framework Changes |
| 433 | +### <!-- id:blob_count_changes/framework --> Framework Changes |
435 | 434 |
|
436 |
| -- [ ] Modify `blob_base_fee_update_fraction`, `target_blobs_per_block`, `max_blobs_per_block` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` if the EIP affects any of the values returned by each function. |
| 435 | +- [ ] <!-- id:blob_count_changes/framework/fork_parameters --> Modify `blob_base_fee_update_fraction`, `target_blobs_per_block`, `max_blobs_per_block` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` if the EIP affects any of the values returned by each function. |
437 | 436 |
|
438 | 437 | ## <!-- id:new_execution_layer_request --> New Execution Layer Request
|
439 | 438 |
|
440 |
| -### <!-- id:new_execution_layer_request_test --> Test Vectors |
| 439 | +### <!-- id:new_execution_layer_request/test --> Test Vectors |
441 | 440 |
|
442 |
| -- [ ] Cross-Request-Type Interaction |
443 |
| - - [ ] Update `tests/prague/eip7685_general_purpose_el_requests` tests to include the new request type in the tests combinations |
| 441 | +- [ ] <!-- id:new_execution_layer_request/test/cross_request_type --> Cross-Request-Type Interaction |
| 442 | + - [ ] <!-- id:new_execution_layer_request/test/cross_request_type/update --> Update `tests/prague/eip7685_general_purpose_el_requests` tests to include the new request type in the tests combinations |
444 | 443 |
|
445 |
| -### <!-- id:new_execution_layer_request_framework --> Framework Changes |
| 444 | +### <!-- id:new_execution_layer_request/framework --> Framework Changes |
446 | 445 |
|
447 |
| -- [ ] Increment `max_request_type` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` to the new maximum request type number after the EIP is activated. |
| 446 | +- [ ] <!-- id:new_execution_layer_request/framework/max_request_type --> Increment `max_request_type` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` to the new maximum request type number after the EIP is activated. |
0 commit comments