You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/checklist_templates/eip_testing_checklist_template.md
+25-29Lines changed: 25 additions & 29 deletions
Original file line number
Diff line number
Diff line change
@@ -5,7 +5,7 @@ Depending on the changes introduced by an EIP, the following template is the min
5
5
## General
6
6
7
7
-[ ] Code coverage
8
-
-[ ] Run produced tests against [EELS](https://github.com/ethereum/execution-specs) and verify that line code coverage of new added lines for the EIP is 100%, with only exeptions being unreachable code lines.
8
+
-[ ] Run produced tests against [EELS](https://github.com/ethereum/execution-specs) and verify that line code coverage of new added lines for the EIP is 100%, with only exceptions being unreachable code lines.
9
9
-[ ] Run coverage on the test code itself (as a basic logic sanity check), i.e., `uv run fill --cov tests`.
10
10
-[ ] Optional - Run against a second client and verify sufficient code coverage over new code added for the EIP.
11
11
-[ ] Fuzzing
@@ -32,9 +32,9 @@ The EIP introduces one or more new opcodes to the EVM.
32
32
-[ ] 2**64 bytes expansion
33
33
-[ ] 2**256-1 bytes expansion (Should always result in Out-of-gas)
34
34
-[ ] Stack
35
-
-[ ]Overflows/Underflows
36
-
- [] If the opcode pushes one or more items to the stack, and the opcode pushes more elements than it pops, verify that the opcode execution results in exeptional abort when pushing elements to the stack would result in the stack having more than 1024 elements.
37
-
- [] If the opcode pops one or more items to the stack, or it has a minimum stack height of one or more (e.g. Op.DUPN requires a minimum amount of elements in the stack even when it does not pop any element from it), verify that the opcode execution results in exeptional abort then stack has 1 less item than the minimum stack height expected.
35
+
-[ ]Overflow/Underflow
36
+
- [] If the opcode pushes one or more items to the stack, and the opcode pushes more elements than it pops, verify that the opcode execution results in exceptional abort when pushing elements to the stack would result in the stack having more than 1024 elements.
37
+
- [] If the opcode pops one or more items to the stack, or it has a minimum stack height of one or more (e.g. Op.DUPN requires a minimum amount of elements in the stack even when it does not pop any element from it), verify that the opcode execution results in exceptional abort then stack has 1 less item than the minimum stack height expected.
38
38
-[ ] If opcode performs stack operations more complex than simple pop/push (e.g. the opcode has a data portion that specifies a variable to access a specific stack element), perform the following test combinations:
39
39
-[ ] Operation on different stack heights:
40
40
- [] Zero (Potential stack-underflow)
@@ -47,9 +47,9 @@ The EIP introduces one or more new opcodes to the EVM.
47
47
-[ ] Execution context
48
48
-[ ] CALL
49
49
-[ ] STATICCALL
50
-
-[ ] Verify exeptional abort if the opcode is banned for static contexts or if it attempts to modify the code, storage or balance of an account.
50
+
-[ ] Verify exceptional abort if the opcode is banned for static contexts or if it attempts to modify the code, storage or balance of an account.
51
51
-[ ] If the opcode is completely banned from static contexts, verify that even when its inputs would not cause any account modification, the opcode still results in exceptional abort of the execution (e.g. Op.PAY with zero value, or Op.SSTORE to the value it already has in the storage).
52
-
-[ ] Verify subcalls using other opcodes (e.g. CALL, DELEGATECALL, etc) also results in the same exeptional abort behavior.
52
+
-[ ] Verify sub-calls using other opcodes (e.g. CALL, DELEGATECALL, etc) also results in the same exceptional abort behavior.
53
53
-[ ] DELEGATECALL
54
54
-[ ] If the opcode modifies the storage of the account currently executing it, verify that only the account that is delegating execution is the one that has its storage modified.
55
55
-[ ] If the opcode modifies the balance of the account currently executing it, verify that only the account that is delegating execution is the one that has its balance modified.
@@ -64,15 +64,15 @@ The EIP introduces one or more new opcodes to the EVM.
64
64
-[ ] Transaction context: If opcode changes behavior depending on particular transaction properties, test using multiple values for each property.
65
65
-[ ] Block context: If opcode changes behavior depending on particular block properties, test using multiple values for each property.
66
66
-[ ] Return data
67
-
-[ ] Verify proper return data buffer overwriting if the opcode is meant to interact with it, otherwise verify that the return data buffer is unnaffected:
67
+
-[ ] Verify proper return data buffer overwriting if the opcode is meant to interact with it, otherwise verify that the return data buffer is unaffected:
68
68
- [] At current call context.
69
69
- [] At parent call context.
70
70
-[ ] Gas usage
71
71
-[ ] Normal operation gas usage: Verify gas affectation of each parameter value consumed by the opcode.
72
72
-[ ] Memory expansion: Verify that the memory expansion correctly follows the gas calculation
73
-
-[ ] Out-of-gas during opcode execution: Verify that attempting to execute the opcode when gas available is 1 less than the required gas results in exeptional abort.
74
-
-[ ] Out-of-gas during memory expansion: Verify that the expansion of memory can result in out-of-gas exeptional abort.
75
-
-[ ] Order-of-operations: If the opcode requires different gas stipends for other operations (e.g. contract creation, cold/warm account access), create one case for each operation (ideally independent of eachother) and the following cases for each:
73
+
-[ ] Out-of-gas during opcode execution: Verify that attempting to execute the opcode when gas available is 1 less than the required gas results in exceptional abort.
74
+
-[ ] Out-of-gas during memory expansion: Verify that the expansion of memory can result in out-of-gas exceptional abort.
75
+
-[ ] Order-of-operations: If the opcode requires different gas stipends for other operations (e.g. contract creation, cold/warm account access), create one case for each operation (ideally independent of each other) and the following cases for each:
76
76
-[ ] Success using the exact amount of gas required for the stipend.
77
77
-[ ] OOG with a 1-gas-difference from the gas required for the stipend.
78
78
-[ ] Terminating opcode
@@ -90,16 +90,16 @@ The EIP introduces one or more new opcodes to the EVM.
90
90
-[ ] Verify if the opcode has out-of-bounds conditions in its parameters and verify:
91
91
- [] Max value for each parameter
92
92
- [] Max value + 1 for each parameter
93
-
-[ ]Exeptional Abort: Verify behavior that is supposed to cause an exeptional abort, other than stack over or underflow, or out-of-gas errors.
93
+
-[ ]Exceptional Abort: Verify behavior that is supposed to cause an exceptional abort, other than stack over or underflow, or out-of-gas errors.
94
94
-[ ] Data portion: If an opcode has data portion that affects its behavior, verify checklist items with multiple interesting values (E.g. if data portion size is 1 byte, use at least 0x00, 0x01, 0x7F, 0xFF).
95
95
-[ ] Contract creation
96
96
-[ ] Verify contract is created at the expected address given multiple inputs to the opcode parameters.
97
97
-[ ] Verify that contract is not created in case of:
98
98
-[ ] Out-of-gas when available gas is less than minimum contract creation stipend.
99
-
-[ ] Creation would result in an address collision with an existing contract or eoa-delegated address.
99
+
-[ ] Creation would result in an address collision with an existing contract or EOA-delegated address.
100
100
-[ ] Verify recursive contract creation using the opcode: Factory contract uses the opcode, and initcode calls back to factory contract.
101
101
-[ ] Fork transition
102
-
-[ ] Verify that the opcode results in exeptional abort if executed before its activation fork.
102
+
-[ ] Verify that the opcode results in exceptional abort if executed before its activation fork.
103
103
-[ ] Verify correct opcode behavior at transition block, in the case of opcodes which behavior depends on current or parent block information.
-[ ] Add precompile address to relevant methods in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py`
@@ -203,8 +202,8 @@ The EIP removes one or more precompiles from the existing list of precompiles.
203
202
-[ ] Verify all zeros input.
204
203
-[ ] Verify 2^N-1 where N is a single or multiple valid bit-lengths.
205
204
-[ ] Verify combinations of invalid inputs to the precompile.
206
-
-[ ] Inputs that fail specific validity checks.
207
-
-[ ] Inputs that are malformed/corrupted.
205
+
-[ ] Inputs that fail specific validity checks.
206
+
-[ ] Inputs that are malformed/corrupted.
208
207
-[ ] Value Transfer
209
208
-[ ] If the system contract requires a minimum value with the calls to it, either constant or depending on a formula, verify:
210
209
- [] Calls with the required value amount minus one, expect failure.
@@ -229,9 +228,8 @@ The EIP removes one or more precompiles from the existing list of precompiles.
229
228
-[ ] If possible, simulate a scenario where the execution of the contract at the end of the block execution by the system address would result in excessive gas usage (100 million gas or more).
230
229
-[ ] Verify spending all block gas in calls to system contract (100 million gas or more).
231
230
-[ ] System Contract Deployment
232
-
-[ ] Verify block execution behavior after fork activation if the system contract has not been deployed.
231
+
-[ ] Verify block execution behavior after fork activation if the system contract has not been deployed (Depending on the EIP, block could be invalid).
233
232
-[ ] Verify deployment transaction results in the system contract being deployed to the expected address.
234
-
<!-- TODO: Missing contract at fork check -->
235
233
-[ ] Contract Variations
236
234
-[ ] Verify execution of the different variations of the contract for different networks (if any) results in the expected behavior,
237
235
-[ ] Verify execution of a variation that causes an exception.
@@ -263,7 +261,7 @@ The EIP removes one or more precompiles from the existing list of precompiles.
263
261
-[ ] Nonce: Verify the transaction (and the block it is included in) is invalid if its nonce value does not match the account's current nonce.
264
262
-[ ] To: Verify the transaction (and the block it is included in) is invalid if the transaction type does not allow contract creation and the to-address field is empty.
265
263
-[ ] Value: Verify the transaction (and the block it is included in) is invalid if the transaction contains a value of 1 and the account does not contain enough funds to cover the intrinsic transaction cost plus 1.
266
-
-[ ] Data: Verify the transaction (and the block it is included in) is invalid if the transaction contains enough data so the data floor cost is higher than the intrinsic gas cost and the gas_limit is equal to the intrinsic gas gost.
264
+
-[ ] Data: Verify the transaction (and the block it is included in) is invalid if the transaction contains enough data so the data floor cost is higher than the intrinsic gas cost and the gas_limit is equal to the intrinsic gas cost.
267
265
-[ ] Sender balance: Verify the transaction (and the block it is included in) is invalid when the sender account does not have enough balance to cover the gas limit multiplied by the max fee per gas.
268
266
-[ ] Signature:
269
267
-[ ] Verify the transaction is correctly rejected if it contains an invalid signature:
@@ -321,28 +319,27 @@ The EIP removes one or more precompiles from the existing list of precompiles.
321
319
-[ ] Verify that the sender account of the new transaction type transaction has its nonce incremented at least by one after the transaction is included in a block (or more if the transaction type introduces a new mechanism that bumps the nonce by more than one).
322
320
-[ ] Verify that the sender account of the new transaction type transaction has its balance reduced by the correct amount (gas consumed and value) at the start of execution (e.g. using Op.BALANCE).
323
321
-[ ] Block Level Interactions
324
-
-[ ] Verify the new transaction type and the following accept/reject behavior depending on interactions with the block gas limit:
322
+
-[ ] Verify the new transaction type and the following accept/reject behavior depending on interactions with the block gas limit:
325
323
- [] New transaction type is the single transaction in the block:
326
324
- [] Rejected if `tx.gas_limit == block.gas_limit + 1`
327
325
- [] Accepted if `tx.gas_limit == block.gas_limit`
328
326
- [] New transaction type is the last transaction of a block with two transactions:
329
327
- [] Accepted if `block.txs[0].gas_used + block.txs[1].gas_limit == block.gas_limit`
330
328
- [] Rejected if `(block.txs[0].gas_used + block.txs[1].gas_limit == block.gas_limit + 1) and (block.txs[0].gas_used < block.gas_limit)`
331
-
-[ ] Verify a transaction of the new type is rejected if its gas limit exceeds the [EIP-7825](https://eips.ethereum.org/EIPS/eip-7825) gas limit for the current fork.
332
-
-[ ] Verify a block with all transactions types including the new type is executed correctly.
329
+
-[ ] Verify a transaction of the new type is rejected if its gas limit exceeds the [EIP-7825](https://eips.ethereum.org/EIPS/eip-7825) gas limit for the current fork.
330
+
-[ ] Verify a block with all transactions types including the new type is executed correctly.
333
331
-[ ] Fork transition: Verify that a block prior to fork activation where the new transaction type is introduced and containing the new transaction type is invalid.
334
332
-[ ] RPC Tests
335
-
-[ ]*Verify `eth_estimateGas` behavior for different valid combinations of the new transaction type
333
+
-[ ]*Verify `eth_estimateGas` behavior for different valid combinations of the new transaction type
336
334
-[ ] Verify `eth_sendRawTransaction` using `execute`
337
335
338
-
*Tests must be added to [`execution-apis`](https://github.com/ethereum/execution-apis) repository.
336
+
*Tests must be added to [`execution-apis`](https://github.com/ethereum/execution-apis) repository.
-[ ] Modify `transaction_intrinsic_cost_calculator` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py`, adding the appropriate new fields that the transaction introduced and the logic to the intrinsic gas cost calculation, if any.
343
341
-[ ] Add the transaction type number to `tx_types` response in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` (If applicable add also to `contract_creating_tx_types`).
344
342
345
-
346
343
## <!-- id:new_block_header_field --> New Block Header Field
347
344
348
345
### <!-- id:new_block_header_field/test --> Test Vectors
@@ -356,7 +353,6 @@ The EIP removes one or more precompiles from the existing list of precompiles.
356
353
-[ ] Verify that a block containing the new header field before the activation of the fork is invalid.
357
354
-[ ] Verify that a block lacking the new header field at the activation of the fork is invalid.
-[ ] Increment `max_request_type` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` to the new maximum request type number after the EIP is activated.
447
+
-[ ] Increment `max_request_type` in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py` to the new maximum request type number after the EIP is activated.
Copy file name to clipboardExpand all lines: docs/writing_tests/post_mortems.md
+2-1Lines changed: 2 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -9,6 +9,7 @@ Each entry must include an explanation of why the test case was missed plus the
9
9
## List
10
10
11
11
## TEMPLATE
12
+
12
13
## Date - Title - Fork
13
14
14
15
### Description
@@ -48,5 +49,5 @@ IDs of the tests added that now cover the missed scenario and link to the docume
48
49
Note any modifications that were introduced in the framework and/or documentation to prevent similar misses.
49
50
50
51
*Example:*
51
-
- Updated EIP checklist to include testing combinations of interesting points related to the elliptic-curve under test, and all combinations between them.
52
52
53
+
- Updated EIP checklist to include testing combinations of interesting points related to the elliptic-curve under test, and all combinations between them.
0 commit comments