You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/checklist_templates/eip_testing_checklist_template.md
+95-21Lines changed: 95 additions & 21 deletions
Original file line number
Diff line number
Diff line change
@@ -6,6 +6,7 @@ Depending on the changes introduced by an EIP, the following template is the min
6
6
7
7
-[ ] Code coverage
8
8
-[ ] Run produced tests against [EELS](https://github.com/ethereum/execution-specs) and verify that line code coverage of new added lines for the EIP is 100%, with only exeptions being unreachable code lines.
9
+
-[ ] Run coverage on the test code itself (as a basic logic sanity check), i.e., `uv run fill --cov tests`.
9
10
-[ ] Optional - Run against a second client and verify sufficient code coverage over new code added for the EIP.
10
11
-[ ] Fuzzing
11
12
-[ ] TBD
@@ -29,16 +30,25 @@ The EIP introduces one or more new opcodes to the EVM.
29
30
-[ ] 2**32 bytes expansion
30
31
-[ ] 2**64-1 bytes expansion
31
32
-[ ] 2**64 bytes expansion
32
-
-[ ] 2**256-1 bytes expansion
33
+
-[ ] 2**256-1 bytes expansion (Should always result in Out-of-gas)
33
34
-[ ] Stack
34
35
-[ ] Overflows/Underflows
35
36
- [] If the opcode pushes one or more items to the stack, and the opcode pushes more elements than it pops, verify that the opcode execution results in exeptional abort when pushing elements to the stack would result in the stack having more than 1024 elements.
36
-
- [] If the opcode pops one or more items to the stack, or it has a minimum stack height of one or more, verify that the opcode execution results in exeptional abort then stack has 1 less item than the minimum stack height expected.
37
-
-[ ] If opcode performs stack operations different of simple pop->push, test for these operations on an asymmetrical stack.
37
+
- [] If the opcode pops one or more items to the stack, or it has a minimum stack height of one or more (e.g. Op.DUPN requires a minimum amount of elements in the stack even when it does not pop any element from it), verify that the opcode execution results in exeptional abort then stack has 1 less item than the minimum stack height expected.
38
+
-[ ] If opcode performs stack operations more complex than simple pop/push (e.g. the opcode has a data portion that specifies a variable to access a specific stack element), perform the following test combinations:
39
+
-[ ] Operation on different stack heights:
40
+
- [] Zero (Potential stack-underflow)
41
+
- [] Odd
42
+
- [] Even
43
+
-[ ] For each variable `n` of the opcode that accesses the nth stack item, test `n` being:
44
+
-[ ] Top stack item
45
+
-[ ] Bottom stack item
46
+
-[ ] Middle stack item
38
47
-[ ] Execution context
39
48
-[ ] CALL
40
49
-[ ] STATICCALL
41
50
-[ ] Verify exeptional abort if the opcode is banned for static contexts or if it attempts to modify the code, storage or balance of an account.
51
+
-[ ] If the opcode is completely banned from static contexts, verify that even when its inputs would not cause any account modification, the opcode still results in exceptional abort of the execution (e.g. Op.PAY with zero value, or Op.SSTORE to the value it already has in the storage).
42
52
-[ ] Verify subcalls using other opcodes (e.g. CALL, DELEGATECALL, etc) also results in the same exeptional abort behavior.
43
53
-[ ] DELEGATECALL
44
54
-[ ] If the opcode modifies the storage of the account currently executing it, verify that only the account that is delegating execution is the one that has its storage modified.
@@ -49,6 +59,8 @@ The EIP introduces one or more new opcodes to the EVM.
49
59
-[ ] Verify opcode behaves as expected when running during the initcode phase of contract creation
50
60
- [] Initcode of a contract creating transaction.
51
61
- [] Initcode of a contract creating opcode (including itself if opcode creates a contract).
62
+
-[ ] Verify opcode behavior on re-entry using the same initcode and same address (e.g. CREATE2->REVERT->CREATE2).
63
+
-[ ] Set-code delegated account: Verify opcode operations are applied to the set-code account and do not affect the address that is the target of the delegation.
52
64
-[ ] Transaction context
53
65
-[ ] If opcode changes behavior depending on particular transaction properties, test using multiple values for each property.
54
66
-[ ] Block context
@@ -58,14 +70,12 @@ The EIP introduces one or more new opcodes to the EVM.
58
70
- [] At current call context.
59
71
- [] At parent call context.
60
72
-[ ] Gas usage
61
-
-[ ] Normal operation gas usage
62
-
- [] Verify gas affectation of each parameter value consumed by the opcode.
63
-
-[ ] Memory expansion
64
-
-[ ] Verify that the memory expansion correctly follows the gas calculation
65
-
-[ ] Out-of-gas during opcode execution
66
-
-[ ] Verify that attempting to execute the opcode when gas available is 1 less than the required gas results in exeptional abort.
67
-
-[ ] Out-of-gas during memory expansion
68
-
-[ ] Verify that the expansion of memory can result in out-of-gas exeptional abort.
73
+
-[ ] Normal operation gas usage: Verify gas affectation of each parameter value consumed by the opcode.
74
+
-[ ] Memory expansion: Verify that the memory expansion correctly follows the gas calculation
75
+
-[ ] Out-of-gas during opcode execution: Verify that attempting to execute the opcode when gas available is 1 less than the required gas results in exeptional abort.
76
+
-[ ] Out-of-gas during memory expansion: Verify that the expansion of memory can result in out-of-gas exeptional abort.
77
+
-[ ] Order-of-operations:
78
+
-[ ] If the opcode requires different gas stipends for other operations (e.g. contract creation, cold/warm account access), create one case for each operation (ideally independent of eachother) and create a case for success and another for OOG (with a 1-gas-difference).
69
79
-[ ] Terminating opcode
70
80
-[ ] If an opcode is terminating, meaning it results in the current call context to end execution, test the following scenarios
71
81
- [] Top-level call termination
@@ -102,6 +112,8 @@ The EIP introduces one or more new opcodes to the EVM.
102
112
103
113
## <!-- id:new_precompile --> New Precompile
104
114
115
+
The EIP introduces one or more new precompiles.
116
+
105
117
### <!-- id:new_precompile/test --> Test Vectors
106
118
107
119
-[ ] Call contexts
@@ -117,9 +129,18 @@ The EIP introduces one or more new opcodes to the EVM.
117
129
-[ ] Inputs
118
130
-[ ] Verify combinations of valid inputs to the precompile
119
131
- [] Verify interesting edge values given the precompile functionality.
120
-
- [] If precompile performs cryptographic operations, verify behavior on all inputs that have special cryptographic properties.
132
+
- [] If precompile performs cryptographic operations, verify behavior on all inputs that have special cryptographic properties (e.g. infinity points as inputs, or input values that result in infinity points returned).
121
133
-[ ] Verify all zeros input.
122
134
-[ ] Verify 2^N-1 where N is a single or multiple valid bit-lengths.
135
+
-[ ] Verify combinations of invalid inputs to the precompile.
136
+
-[ ] Inputs that fail specific mathematical or cryptographic validity checks.
137
+
-[ ] Inputs that are malformed/corrupted.
138
+
-[ ] Value Transfer
139
+
-[ ] If the precompile requires a minimum value with the calls to it, either constant or depending on a formula, verify:
140
+
- [] Calls with the required value amount minus one, expect failure.
141
+
- [] Calls with the exact required amount, expect success.
142
+
- [] Calls with extra value than the required amount, expect success.
143
+
-[ ] If the system contract does not require a minimum value embedded in the calls to it, verify sending value does not cause an exception, unless otherwise specified by the EIP.
123
144
-[ ] Out-of-bounds checks
124
145
-[ ] Verify if the precompile has out-of-bounds conditions in its inputs and verify:
125
146
- [] Max value for each input
@@ -142,7 +163,7 @@ The EIP introduces one or more new opcodes to the EVM.
142
163
-[ ] Verify exact gas consumption, given different valid inputs.
143
164
-[ ] Verify exact gas consumption minus one results in out-of-gas error, given different valid inputs.
144
165
-[ ] Excessive Gas Cases
145
-
-[ ] Verify spending all block gas in calls to the precompile (100 million gas or more).
166
+
-[ ] Verify spending all block gas in calls to the precompile (Use `Environment().gas_limit` as max amount).
146
167
-[ ] Fork transition
147
168
-[ ] Verify that calling the precompile before its activation fork results in a valid call even for inputs that are expected to be invalid for the precompile.
148
169
-[ ] Verify that calling the precompile before its activation fork with zero gas results in a valid call.
@@ -153,6 +174,20 @@ The EIP introduces one or more new opcodes to the EVM.
153
174
154
175
-[ ] Add precompile address to relevant methods in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py`
155
176
177
+
## <!-- id:new_precompile --> Removed Precompile
178
+
179
+
The EIP removes one or more precompiles from the existing list of precompiles.
180
+
181
+
### <!-- id:new_precompile/test --> Test Vectors
182
+
183
+
-[ ] Fork boundary
184
+
-[ ] Verify that the precompile remains operational up until the last block before the fork is active, and behaves as an account with empty code afterwards.
185
+
-[ ] Verify the account is warm up until the last block before the fork is active, and becomes cold afterwards.
-[ ] Remove the precompile address from relevant methods in the fork where the EIP is removed in `src/ethereum_test_forks/forks/forks.py`
190
+
156
191
## <!-- id:new_system_contract --> New System Contract
157
192
158
193
### <!-- id:new_system_contract/test --> Test Vectors
@@ -171,6 +206,15 @@ The EIP introduces one or more new opcodes to the EVM.
171
206
- [] Verify interesting edge values given the system contract functionality.
172
207
-[ ] Verify all zeros input.
173
208
-[ ] Verify 2^N-1 where N is a single or multiple valid bit-lengths.
209
+
-[ ] Verify combinations of invalid inputs to the precompile.
210
+
-[ ] Inputs that fail specific validity checks.
211
+
-[ ] Inputs that are malformed/corrupted.
212
+
-[ ] Value Transfer
213
+
-[ ] If the system contract requires a minimum value with the calls to it, either constant or depending on a formula, verify:
214
+
- [] Calls with the required value amount minus one, expect failure.
215
+
- [] Calls with the exact required amount, expect success.
216
+
- [] Calls with extra value than the required amount, expect success.
217
+
-[ ] If the system contract does not require a minimum value embedded in the calls to it, verify sending value does not cause an exception, unless otherwise specified by the EIP.
174
218
-[ ] Out-of-bounds checks
175
219
-[ ] Verify if the system contract has out-of-bounds conditions in its inputs and verify:
176
220
- [] Max value for each input
@@ -191,6 +235,7 @@ The EIP introduces one or more new opcodes to the EVM.
191
235
-[ ] System Contract Deployment
192
236
-[ ] Verify block execution behavior after fork activation if the system contract has not been deployed.
193
237
-[ ] Verify deployment transaction results in the system contract being deployed to the expected address.
238
+
<!-- TODO: Missing contract at fork check -->
194
239
-[ ] Contract Variations
195
240
-[ ] Verify execution of the different variations of the contract for different networks (if any) results in the expected behavior
196
241
-[ ] Contract Substitution
@@ -258,7 +303,10 @@ The EIP introduces one or more new opcodes to the EVM.
258
303
- []`SECP256K1N - S` of a valid signature
259
304
- Transaction-Scoped Attributes/Variables
260
305
- Verify attributes that can be read in the EVM from transaction fields.
261
-
- Verify values or variables that are persistent through the execution of the transaction (transient storage, warm/cold accounts).
306
+
- Verify attributes specific to the new transaction type that can be read in the EVM behave correctly on older transaction types.
307
+
- Verify values or variables that are persistent through the execution of the transaction (transient storage, warm/cold accounts):
308
+
- Persist throughout the entire transaction.
309
+
- Reset on subsequent transactions in the same block.
262
310
-[ ] Encoding (RLP, SSZ)
263
311
-[ ] Verify correct transaction rejection due to incorrect field sizes:
264
312
- [] Add leading zero byte
@@ -280,8 +328,18 @@ The EIP introduces one or more new opcodes to the EVM.
280
328
-[ ] Contract creation
281
329
-[ ] Verify that the transaction can create new contracts if the transaction type supports it.
282
330
-[ ] Sender account modifications
283
-
-[ ] Verify that the sender account of the new transaction type transaction has its nonce incremented at least by one after the transaction is included in a block (or more if the transaction type introduces a new mechanic that bumps the nonce by more than one).
284
-
-[ ] Verify that the sender account of the new transaction type transaction has its balance reduced by the correct amount (gas consumed and value) after the transaction is included in a block
331
+
-[ ] Verify that the sender account of the new transaction type transaction has its nonce incremented at least by one after the transaction is included in a block (or more if the transaction type introduces a new mechanism that bumps the nonce by more than one).
332
+
-[ ] Verify that the sender account of the new transaction type transaction has its balance reduced by the correct amount (gas consumed and value) at the start of execution (e.g. using Op.BALANCE).
333
+
-[ ] Block Level Interactions
334
+
-[ ] Verify the new transaction type and the following accept/reject behavior depending on interactions with the block gas limit:
335
+
- [] New transaction type is the single transaction in the block:
336
+
- [] Rejected if `tx.gas_limit == block.gas_limit + 1`
337
+
- [] Accepted if `tx.gas_limit == block.gas_limit`
338
+
- [] New transaction type is the last transaction of a block with two transactions:
339
+
- [] Accepted if `block.txs[0].gas_used + block.txs[1].gas_limit == block.gas_limit`
340
+
- [] Rejected if `(block.txs[0].gas_used + block.txs[1].gas_limit == block.gas_limit + 1) and (block.txs[0].gas_used < block.gas_limit)`
341
+
-[ ] Verify a transaction of the new type is rejected if its gas limit exceeds the [EIP-7825](https://eips.ethereum.org/EIPS/eip-7825) gas limit for the current fork.
342
+
-[ ] Verify a block with all transactions types including the new type is executed correctly.
285
343
-[ ] Fork transition
286
344
-[ ] Verify that a block prior to fork activation where the new transaction type is introduced and containing the new transaction type is invalid.
287
345
-[ ] RPC Tests
@@ -303,8 +361,8 @@ The EIP introduces one or more new opcodes to the EVM.
303
361
-[ ] Genesis value
304
362
-[ ] Verify, if possible, that the value can be set at genesis if the network starting fork is the activation fork, and that clients can consume such genesis.
305
363
-[ ] Value behavior
306
-
-[ ] Verify, given multiple initial values, that a block is accepted if the value is correctly modified for the current block, depending on the circumstances that affect the value as defined in the EIP.
307
-
-[ ] Verify, given multiple initial values, that a block is rejected if the value is incorrectly modified for the current block, depending on the circumstances that affect the value as defined in the EIP.
364
+
-[ ] Verify, given multiple initial values, that a block is accepted if the value is the correct expected for the current block, depending on the circumstances that affect the value as defined in the EIP.
365
+
-[ ] Verify, given multiple initial values, that a block is rejected if the value is modified (using `block.rlp_modifier`) to an incorrect value for the current block, depending on the circumstances that affect the value as defined in the EIP.
308
366
-[ ] Fork transition
309
367
-[ ] Verify initial value of the field at the first block of the activation fork.
310
368
-[ ] Verify that a block containing the new header field before the activation of the fork is invalid.
@@ -313,17 +371,33 @@ The EIP introduces one or more new opcodes to the EVM.
-[ ] Add the appropriate `header_*_required` fork method to `BaseFork` in `ethereum_test_forks`.
317
379
318
380
## <!-- id:new_block_body_field --> New Block Body Field
319
381
320
382
### Test Vectors
321
383
322
-
**TBD**
384
+
-[ ] Value behavior
385
+
-[ ] Verify, given multiple initial values, that a block is accepted if the value is the correct expected for the current block, depending on the circumstances that affect the value as defined in the EIP.
386
+
-[ ] Verify, given multiple initial values, that a block is rejected if the value is modified (using appropriate `block`) to an incorrect value for the current block, depending on the circumstances that affect the value as defined in the EIP.
387
+
-[ ] Fork transition
388
+
-[ ] Verify that a block containing the new block body field before the activation of the fork is invalid.
389
+
-[ ] Verify that a block lacking the new block field at the activation of the fork is invalid.
323
390
324
391
### Framework Changes
325
392
326
-
**TBD**
393
+
-[ ] Value behavior
394
+
-[ ] Verify, given multiple initial values, that a block is accepted if the value is correctly modified for the current block, depending on the circumstances that affect the value as defined in the EIP.
395
+
-[ ] Verify, given multiple initial values, that a block is rejected if the value is incorrectly modified for the current block, depending on the circumstances that affect the value as defined in the EIP.
396
+
-[ ] Add the new body field to the relevant objects:
0 commit comments