Skip to content

Commit cc9a56a

Browse files
marioevzdanceratopzspencer-tbjochem-brouwer
authored
feat(docs): Add EIP checklist templates, post-mortem page (#1327)
* docs: Add first checklist templates * changes: Precompiles * changes: System contracts * nit * changes: Add (incomplete) transaction tests * More changes * Add more encoding tests * More cases * More test vectors * More review comments * Add asymetrical stack tests * Add some IDs * Add serialization items * Add more serialization items * Add gas refunds category * docs: Add post-mortems * fix(github): Add the post-mortem docs to the PR template * fix: Remove references to EOF * fix: Review comments Co-authored-by: danceratopz <danceratopz@gmail.com> Co-authored-by: spencer-tb <spencer@spencertaylorbrown.uk> Co-authored-by: Jochem Brouwer <jochembrouwer96@gmail.com> * Add gas tests * Simplify bullet points * fix(docs): tox * Add more ids * Add more ids * Add more ids * Add more ids * fix: tox * refactor(docs): Convert EIP checklist to table * fixes * Eip checklist suggestions (#1601) * chore(docs): auto-format doc, in particular tables, with prettier * chore(docs): remove periods from sub-titles * chore(docs): fix indentation in unordered lists * chore(docs): add missing md table headers * refactor(docs): indent navigation ready for more deeply nested entries * docs: add testing checklists to docs/writing; include in html * docs: update whitelist * docs: disable MD001 for eip testing checklist * chore(docs): more consistent display of opcodes; fix capitalization * fix(docs): Add fuzzing snippet * docs: Changelog --------- Co-authored-by: danceratopz <danceratopz@gmail.com> Co-authored-by: spencer-tb <spencer@spencertaylorbrown.uk> Co-authored-by: Jochem Brouwer <jochembrouwer96@gmail.com>
1 parent 7f2be17 commit cc9a56a

File tree

7 files changed

+1283
-48
lines changed

7 files changed

+1283
-48
lines changed

‎.github/PULL_REQUEST_TEMPLATE.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,4 @@
1414
- [ ] Tests: A PR with removal of converted JSON/YML tests from [ethereum/tests](/ethereum/tests) have been opened.
1515
- [ ] Tests: Included the type and version of evm t8n tool used to locally execute test cases: e.g., ref with commit hash or geth 1.13.1-stable-3f40e65.
1616
- [ ] Tests: Ran `mkdocs serve` locally and verified the auto-generated docs for new tests in the [Test Case Reference](https://eest.ethereum.org/main/tests/) are correctly formatted.
17+
- [ ] Tests: For PRs implementing a missed test case, update the [post-mortem document](/ethereum/execution-spec-tests/blob/main/docs/writing_tests/post_mortems.md) to add an entry the list.

‎docs/CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,9 @@ Test fixtures for use by clients are available for each release on the [Github r
1919

2020
### 📋 Misc
2121

22+
- ✨ Added the [EIP checklist template](https://eest.ethereum.org/main/writing_tests/checklist_templates/eip_testing_checklist_template/) that serves as a reference to achieve better coverage when implementing tests for new EIPs ([#1327](https://github.com/ethereum/execution-spec-tests/pull/1327)).
23+
- ✨ Added [Post-Mortems of Missed Test Scenarios](https://eest.ethereum.org/main/writing_tests/post_mortems/) to the documentation that serves as a reference list of all cases that were missed during the test implementation phase of a new EIP, and includes the steps taken in order to prevent similar test cases to be missed in the future ([#1327](https://github.com/ethereum/execution-spec-tests/pull/1327)).
24+
2225
### 🧪 Test Cases
2326

2427
## [v4.5.0](https://github.com/ethereum/execution-spec-tests/releases/tag/v4.5.0) - 2025-05-14

‎docs/navigation.md

Lines changed: 50 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -1,57 +1,60 @@
11
<!-- markdownlint-disable MD007 MD041 (MD007=ul-indent MD041=first-line-heading) -->
22
* [Home](index.md)
33
* Getting Started
4-
* [Installation](getting_started/installation.md)
5-
* [Installation Troubleshooting](getting_started/installation_troubleshooting.md)
6-
* [VS Code Setup](getting_started/setup_vs_code.md)
7-
* [Repository Overview](getting_started/repository_overview.md)
8-
* [Getting Help](getting_started/getting_help.md)
4+
* [Installation](getting_started/installation.md)
5+
* [Installation Troubleshooting](getting_started/installation_troubleshooting.md)
6+
* [VS Code Setup](getting_started/setup_vs_code.md)
7+
* [Repository Overview](getting_started/repository_overview.md)
8+
* [Getting Help](getting_started/getting_help.md)
99
* [Writing Tests](writing_tests/index.md)
10-
* [Adding a New Test](writing_tests/adding_a_new_test.md)
11-
* [Types of Test](writing_tests/types_of_tests.md)
12-
* [Writing a New Test](writing_tests/writing_a_new_test.md)
13-
* [Test Markers](writing_tests/test_markers.md)
14-
* [Referencing an EIP Spec Version](writing_tests/reference_specification.md)
15-
* [Verifying Changes Locally](writing_tests/verifying_changes.md)
16-
* [Code Standards](writing_tests/code_standards.md)
17-
* [Exception Tests](writing_tests/exception_tests.md)
18-
* [Tutorial: Adding a State Test](writing_tests/tutorials/state_transition.md)
10+
* [Adding a New Test](writing_tests/adding_a_new_test.md)
11+
* [Types of Test](writing_tests/types_of_tests.md)
12+
* [Writing a New Test](writing_tests/writing_a_new_test.md)
13+
* [Test Markers](writing_tests/test_markers.md)
14+
* [Verifying Changes Locally](writing_tests/verifying_changes.md)
15+
* [Code Standards](writing_tests/code_standards.md)
16+
* [Exception Tests](writing_tests/exception_tests.md)
17+
* [Referencing an EIP Spec Version](writing_tests/reference_specification.md)
18+
* [Testing Checklist Templates](writing_tests/checklist_templates/index.md)
19+
* [EIP Execution Layer Testing Checklist Template](writing_tests/checklist_templates/eip_testing_checklist_template.md)
20+
* [Post-mortems](writing_tests/post_mortems.md)
21+
* [Tutorial: Adding a State Test](writing_tests/tutorials/state_transition.md)
1922
* [Filling Tests](filling_tests/index.md)
20-
* [Getting Started](filling_tests/getting_started.md)
21-
* [Filling Tests at a Prompt](filling_tests/filling_tests_command_line.md)
22-
* [Filling Tests in VS Code](filling_tests/filling_tests_vs_code.md)
23-
* [Filling Tests for Features Under Development](filling_tests/filling_tests_dev_fork.md)
24-
* [Transition Tool Support](filling_tests/transition_tool_support.md)
25-
* [Debugging Transition Tools](filling_tests/debugging_t8n_tools.md)
23+
* [Getting Started](filling_tests/getting_started.md)
24+
* [Filling Tests at a Prompt](filling_tests/filling_tests_command_line.md)
25+
* [Filling Tests in VS Code](filling_tests/filling_tests_vs_code.md)
26+
* [Filling Tests for Features Under Development](filling_tests/filling_tests_dev_fork.md)
27+
* [Transition Tool Support](filling_tests/transition_tool_support.md)
28+
* [Debugging Transition Tools](filling_tests/debugging_t8n_tools.md)
2629
* [Consuming Tests](consuming_tests/index.md)
27-
* [State Tests](consuming_tests/state_test.md)
28-
* [Blockchain Tests](consuming_tests/blockchain_test.md)
29-
* [Blockchain Engine Tests](consuming_tests/blockchain_test_engine.md)
30-
* [EOF Tests](consuming_tests/eof_test.md)
31-
* [Transaction Tests](consuming_tests/transaction_test.md)
32-
* [Common Types](consuming_tests/common_types.md)
33-
* [Exceptions](consuming_tests/exceptions.md)
30+
* [State Tests](consuming_tests/state_test.md)
31+
* [Blockchain Tests](consuming_tests/blockchain_test.md)
32+
* [Blockchain Engine Tests](consuming_tests/blockchain_test_engine.md)
33+
* [EOF Tests](consuming_tests/eof_test.md)
34+
* [Transaction Tests](consuming_tests/transaction_test.md)
35+
* [Common Types](consuming_tests/common_types.md)
36+
* [Exceptions](consuming_tests/exceptions.md)
3437
* [Executing Tests](executing_tests/index.md)
3538
* [Developer Doc](dev/index.md)
36-
* [Managing Configurations](dev/configurations.md)
37-
* [Interactive Library Usage](dev/interactive_usage.md)
38-
* [Generating Documentation](dev/docs.md)
39-
* [Documenting CLI Commands](dev/documenting_clis.md)
40-
* [Coding Style](dev/coding_style.md)
41-
* [Logging](dev/logging.md)
42-
* [Enabling Precommit Checks](dev/precommit.md)
43-
* [Running Github Actions Locally](dev/test_actions_locally.md)
44-
* [Porting Legacy Tests](dev/porting_legacy_tests.md)
39+
* [Managing Configurations](dev/configurations.md)
40+
* [Interactive Library Usage](dev/interactive_usage.md)
41+
* [Generating Documentation](dev/docs.md)
42+
* [Documenting CLI Commands](dev/documenting_clis.md)
43+
* [Coding Style](dev/coding_style.md)
44+
* [Logging](dev/logging.md)
45+
* [Enabling Precommit Checks](dev/precommit.md)
46+
* [Running Github Actions Locally](dev/test_actions_locally.md)
47+
* [Porting Legacy Tests](dev/porting_legacy_tests.md)
4548
* [Changelog](CHANGELOG.md)
4649
* [Library Reference](library/index.md)
47-
* [EEST CLI Tools](library/cli/index.md)
48-
* [Ethereum Test Base Types Package](library/ethereum_test_base_types.md)
49-
* [Ethereum Test Exceptions Package](library/ethereum_test_exceptions.md)
50-
* [Ethereum Test Fixtures Package](library/ethereum_test_fixtures.md)
51-
* [Ethereum Test Forks Package](library/ethereum_test_forks.md)
52-
* [Ethereum Test Specs Package](library/ethereum_test_specs.md)
53-
* [Ethereum Test Tools Package](library/ethereum_test_tools.md)
54-
* [Ethereum Test Types Package](library/ethereum_test_types.md)
55-
* [Ethereum Test VM Package](library/ethereum_test_vm.md)
56-
* [Ethereum CLIs Package](library/ethereum_clis.md)
57-
* [Pytest Plugins](library/pytest_plugins/index.md)
50+
* [EEST CLI Tools](library/cli/index.md)
51+
* [Ethereum Test Base Types Package](library/ethereum_test_base_types.md)
52+
* [Ethereum Test Exceptions Package](library/ethereum_test_exceptions.md)
53+
* [Ethereum Test Fixtures Package](library/ethereum_test_fixtures.md)
54+
* [Ethereum Test Forks Package](library/ethereum_test_forks.md)
55+
* [Ethereum Test Specs Package](library/ethereum_test_specs.md)
56+
* [Ethereum Test Tools Package](library/ethereum_test_tools.md)
57+
* [Ethereum Test Types Package](library/ethereum_test_types.md)
58+
* [Ethereum Test VM Package](library/ethereum_test_vm.md)
59+
* [Ethereum CLIs Package](library/ethereum_clis.md)
60+
* [Pytest Plugins](library/pytest_plugins/index.md)

0 commit comments

Comments
 (0)