Skip to content

Commit ccab8c1

Browse files
marioevzdanceratopzspencer-tbjochem-brouwer
committed
fix: Review comments
Co-authored-by: danceratopz <danceratopz@gmail.com> Co-authored-by: spencer-tb <spencer@spencertaylorbrown.uk> Co-authored-by: Jochem Brouwer <jochembrouwer96@gmail.com>
1 parent 862009d commit ccab8c1

File tree

1 file changed

+95
-21
lines changed

1 file changed

+95
-21
lines changed

docs/checklist_templates/eip_testing_checklist_template.md

Lines changed: 95 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ Depending on the changes introduced by an EIP, the following template is the min
66

77
- [ ] Code coverage
88
- [ ] Run produced tests against [EELS](https://github.com/ethereum/execution-specs) and verify that line code coverage of new added lines for the EIP is 100%, with only exeptions being unreachable code lines.
9+
- [ ] Run coverage on the test code itself (as a basic logic sanity check), i.e., `uv run fill --cov tests`.
910
- [ ] Optional - Run against a second client and verify sufficient code coverage over new code added for the EIP.
1011
- [ ] Fuzzing
1112
- [ ] TBD
@@ -29,16 +30,25 @@ The EIP introduces one or more new opcodes to the EVM.
2930
- [ ] 2**32 bytes expansion
3031
- [ ] 2**64-1 bytes expansion
3132
- [ ] 2**64 bytes expansion
32-
- [ ] 2**256-1 bytes expansion
33+
- [ ] 2**256-1 bytes expansion (Should always result in Out-of-gas)
3334
- [ ] Stack
3435
- [ ] Overflows/Underflows
3536
- [ ] If the opcode pushes one or more items to the stack, and the opcode pushes more elements than it pops, verify that the opcode execution results in exeptional abort when pushing elements to the stack would result in the stack having more than 1024 elements.
36-
- [ ] If the opcode pops one or more items to the stack, or it has a minimum stack height of one or more, verify that the opcode execution results in exeptional abort then stack has 1 less item than the minimum stack height expected.
37-
- [ ] If opcode performs stack operations different of simple pop->push, test for these operations on an asymmetrical stack.
37+
- [ ] If the opcode pops one or more items to the stack, or it has a minimum stack height of one or more (e.g. Op.DUPN requires a minimum amount of elements in the stack even when it does not pop any element from it), verify that the opcode execution results in exeptional abort then stack has 1 less item than the minimum stack height expected.
38+
- [ ] If opcode performs stack operations more complex than simple pop/push (e.g. the opcode has a data portion that specifies a variable to access a specific stack element), perform the following test combinations:
39+
- [ ] Operation on different stack heights:
40+
- [ ] Zero (Potential stack-underflow)
41+
- [ ] Odd
42+
- [ ] Even
43+
- [ ] For each variable `n` of the opcode that accesses the nth stack item, test `n` being:
44+
- [ ] Top stack item
45+
- [ ] Bottom stack item
46+
- [ ] Middle stack item
3847
- [ ] Execution context
3948
- [ ] CALL
4049
- [ ] STATICCALL
4150
- [ ] Verify exeptional abort if the opcode is banned for static contexts or if it attempts to modify the code, storage or balance of an account.
51+
- [ ] If the opcode is completely banned from static contexts, verify that even when its inputs would not cause any account modification, the opcode still results in exceptional abort of the execution (e.g. Op.PAY with zero value, or Op.SSTORE to the value it already has in the storage).
4252
- [ ] Verify subcalls using other opcodes (e.g. CALL, DELEGATECALL, etc) also results in the same exeptional abort behavior.
4353
- [ ] DELEGATECALL
4454
- [ ] If the opcode modifies the storage of the account currently executing it, verify that only the account that is delegating execution is the one that has its storage modified.
@@ -49,6 +59,8 @@ The EIP introduces one or more new opcodes to the EVM.
4959
- [ ] Verify opcode behaves as expected when running during the initcode phase of contract creation
5060
- [ ] Initcode of a contract creating transaction.
5161
- [ ] Initcode of a contract creating opcode (including itself if opcode creates a contract).
62+
- [ ] Verify opcode behavior on re-entry using the same initcode and same address (e.g. CREATE2->REVERT->CREATE2).
63+
- [ ] Set-code delegated account: Verify opcode operations are applied to the set-code account and do not affect the address that is the target of the delegation.
5264
- [ ] Transaction context
5365
- [ ] If opcode changes behavior depending on particular transaction properties, test using multiple values for each property.
5466
- [ ] Block context
@@ -58,14 +70,12 @@ The EIP introduces one or more new opcodes to the EVM.
5870
- [ ] At current call context.
5971
- [ ] At parent call context.
6072
- [ ] Gas usage
61-
- [ ] Normal operation gas usage
62-
- [ ] Verify gas affectation of each parameter value consumed by the opcode.
63-
- [ ] Memory expansion
64-
- [ ] Verify that the memory expansion correctly follows the gas calculation
65-
- [ ] Out-of-gas during opcode execution
66-
- [ ] Verify that attempting to execute the opcode when gas available is 1 less than the required gas results in exeptional abort.
67-
- [ ] Out-of-gas during memory expansion
68-
- [ ] Verify that the expansion of memory can result in out-of-gas exeptional abort.
73+
- [ ] Normal operation gas usage: Verify gas affectation of each parameter value consumed by the opcode.
74+
- [ ] Memory expansion: Verify that the memory expansion correctly follows the gas calculation
75+
- [ ] Out-of-gas during opcode execution: Verify that attempting to execute the opcode when gas available is 1 less than the required gas results in exeptional abort.
76+
- [ ] Out-of-gas during memory expansion: Verify that the expansion of memory can result in out-of-gas exeptional abort.
77+
- [ ] Order-of-operations:
78+
- [ ] If the opcode requires different gas stipends for other operations (e.g. contract creation, cold/warm account access), create one case for each operation (ideally independent of eachother) and create a case for success and another for OOG (with a 1-gas-difference).
6979
- [ ] Terminating opcode
7080
- [ ] If an opcode is terminating, meaning it results in the current call context to end execution, test the following scenarios
7181
- [ ] Top-level call termination
@@ -102,6 +112,8 @@ The EIP introduces one or more new opcodes to the EVM.
102112

103113
## <!-- id:new_precompile --> New Precompile
104114

115+
The EIP introduces one or more new precompiles.
116+
105117
### <!-- id:new_precompile/test --> Test Vectors
106118

107119
- [ ] Call contexts
@@ -117,9 +129,18 @@ The EIP introduces one or more new opcodes to the EVM.
117129
- [ ] Inputs
118130
- [ ] Verify combinations of valid inputs to the precompile
119131
- [ ] Verify interesting edge values given the precompile functionality.
120-
- [ ] If precompile performs cryptographic operations, verify behavior on all inputs that have special cryptographic properties.
132+
- [ ] If precompile performs cryptographic operations, verify behavior on all inputs that have special cryptographic properties (e.g. infinity points as inputs, or input values that result in infinity points returned).
121133
- [ ] Verify all zeros input.
122134
- [ ] Verify 2^N-1 where N is a single or multiple valid bit-lengths.
135+
- [ ] Verify combinations of invalid inputs to the precompile.
136+
- [ ] Inputs that fail specific mathematical or cryptographic validity checks.
137+
- [ ] Inputs that are malformed/corrupted.
138+
- [ ] Value Transfer
139+
- [ ] If the precompile requires a minimum value with the calls to it, either constant or depending on a formula, verify:
140+
- [ ] Calls with the required value amount minus one, expect failure.
141+
- [ ] Calls with the exact required amount, expect success.
142+
- [ ] Calls with extra value than the required amount, expect success.
143+
- [ ] If the system contract does not require a minimum value embedded in the calls to it, verify sending value does not cause an exception, unless otherwise specified by the EIP.
123144
- [ ] Out-of-bounds checks
124145
- [ ] Verify if the precompile has out-of-bounds conditions in its inputs and verify:
125146
- [ ] Max value for each input
@@ -142,7 +163,7 @@ The EIP introduces one or more new opcodes to the EVM.
142163
- [ ] Verify exact gas consumption, given different valid inputs.
143164
- [ ] Verify exact gas consumption minus one results in out-of-gas error, given different valid inputs.
144165
- [ ] Excessive Gas Cases
145-
- [ ] Verify spending all block gas in calls to the precompile (100 million gas or more).
166+
- [ ] Verify spending all block gas in calls to the precompile (Use `Environment().gas_limit` as max amount).
146167
- [ ] Fork transition
147168
- [ ] Verify that calling the precompile before its activation fork results in a valid call even for inputs that are expected to be invalid for the precompile.
148169
- [ ] Verify that calling the precompile before its activation fork with zero gas results in a valid call.
@@ -153,6 +174,20 @@ The EIP introduces one or more new opcodes to the EVM.
153174

154175
- [ ] Add precompile address to relevant methods in the fork where the EIP is introduced in `src/ethereum_test_forks/forks/forks.py`
155176

177+
## <!-- id:new_precompile --> Removed Precompile
178+
179+
The EIP removes one or more precompiles from the existing list of precompiles.
180+
181+
### <!-- id:new_precompile/test --> Test Vectors
182+
183+
- [ ] Fork boundary
184+
- [ ] Verify that the precompile remains operational up until the last block before the fork is active, and behaves as an account with empty code afterwards.
185+
- [ ] Verify the account is warm up until the last block before the fork is active, and becomes cold afterwards.
186+
187+
### <!-- id:new_precompile/framework --> Framework Changes
188+
189+
- [ ] Remove the precompile address from relevant methods in the fork where the EIP is removed in `src/ethereum_test_forks/forks/forks.py`
190+
156191
## <!-- id:new_system_contract --> New System Contract
157192

158193
### <!-- id:new_system_contract/test --> Test Vectors
@@ -171,6 +206,15 @@ The EIP introduces one or more new opcodes to the EVM.
171206
- [ ] Verify interesting edge values given the system contract functionality.
172207
- [ ] Verify all zeros input.
173208
- [ ] Verify 2^N-1 where N is a single or multiple valid bit-lengths.
209+
- [ ] Verify combinations of invalid inputs to the precompile.
210+
- [ ] Inputs that fail specific validity checks.
211+
- [ ] Inputs that are malformed/corrupted.
212+
- [ ] Value Transfer
213+
- [ ] If the system contract requires a minimum value with the calls to it, either constant or depending on a formula, verify:
214+
- [ ] Calls with the required value amount minus one, expect failure.
215+
- [ ] Calls with the exact required amount, expect success.
216+
- [ ] Calls with extra value than the required amount, expect success.
217+
- [ ] If the system contract does not require a minimum value embedded in the calls to it, verify sending value does not cause an exception, unless otherwise specified by the EIP.
174218
- [ ] Out-of-bounds checks
175219
- [ ] Verify if the system contract has out-of-bounds conditions in its inputs and verify:
176220
- [ ] Max value for each input
@@ -191,6 +235,7 @@ The EIP introduces one or more new opcodes to the EVM.
191235
- [ ] System Contract Deployment
192236
- [ ] Verify block execution behavior after fork activation if the system contract has not been deployed.
193237
- [ ] Verify deployment transaction results in the system contract being deployed to the expected address.
238+
<!-- TODO: Missing contract at fork check -->
194239
- [ ] Contract Variations
195240
- [ ] Verify execution of the different variations of the contract for different networks (if any) results in the expected behavior
196241
- [ ] Contract Substitution
@@ -258,7 +303,10 @@ The EIP introduces one or more new opcodes to the EVM.
258303
- [ ] `SECP256K1N - S` of a valid signature
259304
- Transaction-Scoped Attributes/Variables
260305
- Verify attributes that can be read in the EVM from transaction fields.
261-
- Verify values or variables that are persistent through the execution of the transaction (transient storage, warm/cold accounts).
306+
- Verify attributes specific to the new transaction type that can be read in the EVM behave correctly on older transaction types.
307+
- Verify values or variables that are persistent through the execution of the transaction (transient storage, warm/cold accounts):
308+
- Persist throughout the entire transaction.
309+
- Reset on subsequent transactions in the same block.
262310
- [ ] Encoding (RLP, SSZ)
263311
- [ ] Verify correct transaction rejection due to incorrect field sizes:
264312
- [ ] Add leading zero byte
@@ -280,8 +328,18 @@ The EIP introduces one or more new opcodes to the EVM.
280328
- [ ] Contract creation
281329
- [ ] Verify that the transaction can create new contracts if the transaction type supports it.
282330
- [ ] Sender account modifications
283-
- [ ] Verify that the sender account of the new transaction type transaction has its nonce incremented at least by one after the transaction is included in a block (or more if the transaction type introduces a new mechanic that bumps the nonce by more than one).
284-
- [ ] Verify that the sender account of the new transaction type transaction has its balance reduced by the correct amount (gas consumed and value) after the transaction is included in a block
331+
- [ ] Verify that the sender account of the new transaction type transaction has its nonce incremented at least by one after the transaction is included in a block (or more if the transaction type introduces a new mechanism that bumps the nonce by more than one).
332+
- [ ] Verify that the sender account of the new transaction type transaction has its balance reduced by the correct amount (gas consumed and value) at the start of execution (e.g. using Op.BALANCE).
333+
- [ ] Block Level Interactions
334+
- [ ] Verify the new transaction type and the following accept/reject behavior depending on interactions with the block gas limit:
335+
- [ ] New transaction type is the single transaction in the block:
336+
- [ ] Rejected if `tx.gas_limit == block.gas_limit + 1`
337+
- [ ] Accepted if `tx.gas_limit == block.gas_limit`
338+
- [ ] New transaction type is the last transaction of a block with two transactions:
339+
- [ ] Accepted if `block.txs[0].gas_used + block.txs[1].gas_limit == block.gas_limit`
340+
- [ ] Rejected if `(block.txs[0].gas_used + block.txs[1].gas_limit == block.gas_limit + 1) and (block.txs[0].gas_used < block.gas_limit)`
341+
- [ ] Verify a transaction of the new type is rejected if its gas limit exceeds the [EIP-7825](https://eips.ethereum.org/EIPS/eip-7825) gas limit for the current fork.
342+
- [ ] Verify a block with all transactions types including the new type is executed correctly.
285343
- [ ] Fork transition
286344
- [ ] Verify that a block prior to fork activation where the new transaction type is introduced and containing the new transaction type is invalid.
287345
- [ ] RPC Tests
@@ -303,8 +361,8 @@ The EIP introduces one or more new opcodes to the EVM.
303361
- [ ] Genesis value
304362
- [ ] Verify, if possible, that the value can be set at genesis if the network starting fork is the activation fork, and that clients can consume such genesis.
305363
- [ ] Value behavior
306-
- [ ] Verify, given multiple initial values, that a block is accepted if the value is correctly modified for the current block, depending on the circumstances that affect the value as defined in the EIP.
307-
- [ ] Verify, given multiple initial values, that a block is rejected if the value is incorrectly modified for the current block, depending on the circumstances that affect the value as defined in the EIP.
364+
- [ ] Verify, given multiple initial values, that a block is accepted if the value is the correct expected for the current block, depending on the circumstances that affect the value as defined in the EIP.
365+
- [ ] Verify, given multiple initial values, that a block is rejected if the value is modified (using `block.rlp_modifier`) to an incorrect value for the current block, depending on the circumstances that affect the value as defined in the EIP.
308366
- [ ] Fork transition
309367
- [ ] Verify initial value of the field at the first block of the activation fork.
310368
- [ ] Verify that a block containing the new header field before the activation of the fork is invalid.
@@ -313,17 +371,33 @@ The EIP introduces one or more new opcodes to the EVM.
313371

314372
### <!-- id:new_block_header/framework --> Framework Changes
315373

316-
**TBD**
374+
- [ ] Add the new header field to the relevant objects:
375+
- [ ] `ethereum_test_fixtures.FixtureHeader`
376+
- [ ] `ethereum_test_fixtures.FixtureExecutionPayload`
377+
- [ ] `ethereum_test_specs.Header`
378+
- [ ] Add the appropriate `header_*_required` fork method to `BaseFork` in `ethereum_test_forks`.
317379

318380
## <!-- id:new_block_body_field --> New Block Body Field
319381

320382
### Test Vectors
321383

322-
**TBD**
384+
- [ ] Value behavior
385+
- [ ] Verify, given multiple initial values, that a block is accepted if the value is the correct expected for the current block, depending on the circumstances that affect the value as defined in the EIP.
386+
- [ ] Verify, given multiple initial values, that a block is rejected if the value is modified (using appropriate `block`) to an incorrect value for the current block, depending on the circumstances that affect the value as defined in the EIP.
387+
- [ ] Fork transition
388+
- [ ] Verify that a block containing the new block body field before the activation of the fork is invalid.
389+
- [ ] Verify that a block lacking the new block field at the activation of the fork is invalid.
323390

324391
### Framework Changes
325392

326-
**TBD**
393+
- [ ] Value behavior
394+
- [ ] Verify, given multiple initial values, that a block is accepted if the value is correctly modified for the current block, depending on the circumstances that affect the value as defined in the EIP.
395+
- [ ] Verify, given multiple initial values, that a block is rejected if the value is incorrectly modified for the current block, depending on the circumstances that affect the value as defined in the EIP.
396+
- [ ] Add the new body field to the relevant objects:
397+
- [ ] `ethereum_test_fixtures.FixtureBlockBase`
398+
- [ ] `ethereum_test_fixtures.FixtureEngineNewPayload`
399+
- [ ] `ethereum_test_specs.Block`
400+
- [ ] Modify `ethereum_test_specs.BlockchainTest` filling behavior to account for the new block field.
327401

328402
## <!-- id:gas_cost_changes --> Gas Cost Changes
329403

0 commit comments

Comments
 (0)