Skip to content

Commit 0c00c94

Browse files
danceratopzmarioevzfselmo
authored and
Carsons-Eels
committed
chore(tooling): add CLAUDE.md (ethereum#1749)
Co-authored-by: Mario Vega <marioevz@gmail.com> Co-authored-by: felipe <fselmo2@gmail.com>
1 parent f1f7af6 commit 0c00c94

File tree

3 files changed

+370
-0
lines changed

3 files changed

+370
-0
lines changed

‎CLAUDE.md

Lines changed: 200 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,200 @@
1+
# CLAUDE.md - Ethereum Execution Spec Tests
2+
3+
> **CRITICAL**: This repository aims to provide excellent tooling for generating JSON test vectors that test Ethereum execution layer clients. Correctness is absolute priority. The repo prioritizes a contributor-first mindset.
4+
5+
## 🎯 Core Purpose
6+
7+
- `./tests/`: Python tests that **generate JSON test vectors (fixtures)** via `fill` command
8+
- `./src/pytest_plugins/filler/`: Implements `fill` command (test vector generation from Python source)
9+
- `./src/pytest_plugins/consume/`: Implements `consume` command (test vector execution)
10+
- `./src/pytest_plugins/execute/`: Implements `execute` command (live JSON-RPC testing from Python source)
11+
- `./src/ethereum_test_*`: Core framework libraries and data structures
12+
13+
### Key Terminology (CRITICAL)
14+
15+
**"Fixtures" has TWO meanings:**
16+
17+
1. **Test Fixtures** (JSON files) - The test vectors this framework generates
18+
2. **Pytest Fixtures** - Standard pytest setup/teardown (`pre`, `state_test`, etc.)
19+
20+
### Workflows
21+
22+
```text
23+
Fill/Consume: Python Tests → fill → JSON Fixtures → consume → Client Testing
24+
Execute: Python Tests → execute → Live JSON-RPC Testing
25+
```
26+
27+
## 🚀 Essential Commands
28+
29+
All commands use `uv run` prefix.
30+
31+
### Setup
32+
33+
```bash
34+
uv sync --all-extras
35+
uv run solc-select use 0.8.24 --always-install
36+
uvx pre-commit install
37+
```
38+
39+
### Core Workflow
40+
41+
```bash
42+
# Create test
43+
uv run eest make test
44+
45+
# Generate fixtures (PRIMARY WORKFLOW)
46+
uv run fill --fork=Prague path/to/test.py --clean -v -m "not slow"
47+
48+
# Execute against client
49+
uv run consume direct --bin=evm fixtures/
50+
51+
# Framework testing
52+
uv run pytest -c pytest-framework.ini path/to/test.py::test_function
53+
```
54+
55+
### Quality Checks
56+
57+
```bash
58+
# Check code style and errors
59+
uv run ruff check src tests .github/scripts
60+
61+
# Format code
62+
uv run ruff format src tests .github/scripts
63+
64+
# Fix auto-fixable issues
65+
uv run ruff check --fix src tests .github/scripts
66+
67+
# Type checking
68+
uv run mypy src tests .github/scripts
69+
70+
# Framework unit tests
71+
uv run pytest -c pytest-framework.ini -n auto -m "not run_in_serial"
72+
uv run pytest -c pytest-framework.ini -m run_in_serial
73+
74+
# Run specific checks (fast checks)
75+
uvx --with=tox-uv tox -e lint,typecheck,spellcheck
76+
77+
# Local docs check (fast mode: these warnings can be ignored "WARNING - Doc file 'writing_tests/..."):
78+
export FAST_DOCS=true && export GEN_TEST_DOC_VERSION="tox" && uv run mkdocs build
79+
```
80+
81+
## 🎯 Core Framework Rules
82+
83+
### NEVER Use Hardcoded Addresses
84+
85+
```python
86+
def test_example(pre: Alloc, state_test: StateTestFiller):
87+
# ✅ Dynamic allocation
88+
sender = pre.fund_eoa()
89+
contract = pre.deploy_contract(code=Op.SSTORE(1, 1))
90+
91+
tx = Transaction(sender=sender, to=contract, gas_limit=5_000_000)
92+
state_test(pre=pre, tx=tx, post={contract: Account(storage={1: 1})})
93+
```
94+
95+
### Key Methods
96+
97+
- `pre.deploy_contract(code, **kwargs) -> Address`
98+
- `pre.fund_eoa(amount=None, **kwargs) -> EOA`
99+
- `pre.fund_address(address, amount)`
100+
101+
### Gas Calculation Pattern
102+
103+
```python
104+
intrinsic_gas_calculator = fork.transaction_intrinsic_cost_calculator()
105+
tx_gas_limit = intrinsic_gas_calculator(
106+
calldata=tx_data,
107+
contract_creation=False,
108+
access_list=access_lists,
109+
) + 100_000
110+
```
111+
112+
## 📁 Key Directories
113+
114+
```text
115+
src/
116+
├── ethereum_test_tools/ # Core framework
117+
├── ethereum_test_types/ # Type definitions
118+
├── ethereum_test_fixtures/ # Pydantic models for test fixtures
119+
├── pytest_plugins/ # Plugin system
120+
tests/ # Test cases by fork
121+
fixtures/ # Generated test vectors (default output directory)
122+
```
123+
124+
## ⚠️ Critical Anti-Patterns
125+
126+
- ❌ Hardcoded addresses (use `pre` fixture)
127+
- ❌ `TestAddress` in new tests (use `pre.fund_eoa()`)
128+
- ❌ Missing `sender` parameter in transactions
129+
- ❌ Missing `@pytest.mark.valid_from("Fork")` markers
130+
- ❌ Manual nonce management
131+
132+
## 🔧 Common Patterns
133+
134+
### Fork Compatibility
135+
136+
```python
137+
@pytest.mark.valid_from("Cancun")
138+
def test_example(pre: Alloc, state_test: StateTestFiller):
139+
if fork >= Fork.Cancun:
140+
# Cancun-specific logic
141+
else:
142+
# Pre-Cancun logic
143+
```
144+
145+
### Parameterized Tests
146+
147+
```python
148+
@pytest.mark.parametrize("value", [0, 1, 2**256-1])
149+
def test_with_params(value: int, pre: Alloc, state_test: StateTestFiller):
150+
```
151+
152+
## 🐛 Debugging Test Filling
153+
154+
### Generate EVM Traces
155+
156+
```bash
157+
uv run fill --fork=Prague --evm-dump-dir=debug_output/ --traces path/to/test.py
158+
jq -r '.opName' debug_output/**/*.jsonl
159+
```
160+
161+
### Common Issues
162+
163+
- **Fill failures**: Check gas limits (increase if needed, use `transaction_intrinsic_cost_calculator`)
164+
- **Address conflicts**: Always use `pre` fixture for dynamic allocation
165+
- **Test collection**: Functions must start with `test_`
166+
- **Import errors**: Check dependencies in `pyproject.toml`, run `uv sync --all-extras`
167+
168+
## 📝 Code Standards
169+
170+
- **Line length**: 100 characters
171+
- **Type annotations**: Required
172+
- **Import style**: Explicit imports only, no `import *`, no local imports.
173+
- **Path handling**: Use `pathlib`
174+
- **Code**: Use idiomatic python, prioritize readability.
175+
- **Docstrings**: Always include for methods and classes. For one-liners """Use one single-line docstring with quotes on same line."""
176+
177+
## Commit Format
178+
179+
```text
180+
<type>(<scope>): <description>
181+
182+
# Types: feat, fix, docs, style, refactor, test, chore, new
183+
# Scopes: evm, forks, tools, pytest, tests, docs, ci, consume, fill, eest
184+
```
185+
186+
## 🔍 Tool Preferences
187+
188+
- **Search**: `rg "pattern" --type python` (not grep)
189+
- **JSON**: `jq -r '.field' file.json`
190+
- **GitHub**: `gh` CLI for all operations
191+
192+
## 🎯 Development Workflow
193+
194+
1. `uv run eest make test` - Create test
195+
2. Implement tests using `pre` fixture patterns
196+
3. `uv run fill --fork=Fork test.py --clean -v tests/path/to/module` - Generate fixtures
197+
4. `uv run ruff check --fix` - Fix linting
198+
5. Commit with semantic format
199+
200+
**Critical**: Always run linting and type checking. Use `--clean` when filling. Never use hardcoded addresses.

‎HUMANS.md

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
# HUMANS.md - Working with Claude and LLMs in Ethereum Execution Spec Tests
2+
3+
This guide helps human developers understand the dependencies and get the most out of Claude and other LLMs when working with this codebase.
4+
5+
## 🤖 Why This Repository Has LLM Support
6+
7+
Humans are faster when they use LLMs correctly.
8+
9+
## 📋 Required Dependencies for LLM-Assisted Development
10+
11+
### Requirements
12+
13+
#### LLM Context File
14+
15+
- **CLAUDE.md** - Primary LLM guidance (keep up-to-date). Use `#memorize` in Claude to update with new info.
16+
17+
### Recommended Available Tooling (for use with Claude)
18+
19+
1. **GitHub CLI**: `gh` for PR and issue management.
20+
2. **ripgrep**: `rg` for fast code searching.
21+
3. **jq**: For JSON analysis and EVM trace debugging.
22+
4. **markdownlint-cli**: To verify markdown files (CI enforced).
23+
5. **VS Code**: With recommended extensions (see [setup guide](docs/getting_started/setup_vs_code.md)). Run `claude` in VS Code for the best results.
24+
25+
## 🎯 Getting the Best Results from Claude
26+
27+
### 1. Provide Relevant Context
28+
29+
**Always mention:**
30+
31+
- What you're trying to accomplish.
32+
- Which part of the codebase you're working on (`tests/`, `src/`, `docs/`).
33+
- Any error messages or specific issues you're encountering.
34+
35+
**Example - Good Context**:
36+
> "I'm writing a new test for EIP-7702 in `tests/prague/eip7702_set_code_tx/`. The test should verify that delegation target validation works correctly. The test fails to fill when running `uv run fill --fork=Prague path/to/test.py`"
37+
38+
**Example - Poor Context**:
39+
> "My test isn't working"
40+
41+
### 2. Reference Key Documentation
42+
43+
When asking Claude for help, mention which documentation you've already checked:
44+
45+
- "I've read the test patterns in CLAUDE.md but...".
46+
- "According to the debugging section in CLAUDE.md...".
47+
- "The CONTRIBUTING.md mentions X, but I need help with Y...".
48+
49+
### 3. Share Specific Commands and Output
50+
51+
Claude works best with concrete information:
52+
53+
```bash
54+
# Share the exact command you ran
55+
uv run fill --fork=Prague tests/cancun/eip4844_blobs/test_blob_txs.py --clean -v
56+
57+
# Include relevant error output
58+
ERROR: Failed to compile Yul source: ...
59+
```
60+
61+
### 4. Ask for Complete Solutions
62+
63+
Request end-to-end guidance rather than partial answers:
64+
65+
- "Show me the complete test function with proper imports.".
66+
- "What's the full workflow from creating the test to verifying it works?".
67+
- "Include the commands I need to run to validate this change.".
68+
69+
## 🚀 Optimizing LLM Workflows
70+
71+
### Quick Start Template
72+
73+
When starting a new (and complicated) task, provide Claude with something similar to template.
74+
75+
```console
76+
I'm working on [describe task] in the Ethereum execution-spec-tests repository.
77+
78+
**Context**:
79+
- Working directory: [tests/shanghai/, src/ethereum_test_tools/, etc.]
80+
- Trying to: [specific goal]
81+
- Current status: [what you've tried, any errors]
82+
83+
**References**:
84+
- Checked CLAUDE.md section: [which sections you've read]
85+
- Related documentation: [any other docs you've reviewed]
86+
87+
**Specific question**: [exactly what you need help with]
88+
```
89+
90+
### Debugging Template
91+
92+
For troubleshooting issues:
93+
94+
```console
95+
I'm encountering [specific error] when [doing what].
96+
97+
**Command run**:
98+
```bash
99+
[exact command that failed]
100+
```console
101+
102+
**Error output**:
103+
104+
```
105+
[full error message]
106+
```console
107+
108+
**What I've tried**:
109+
110+
- [list previous attempts].
111+
112+
**Environment details**:
113+
114+
```bash
115+
uv run eest info
116+
```console
117+
118+
**Request**: Please help me understand what's wrong and provide the fix.
119+
120+
```
121+
122+
### Code Review Template
123+
124+
When asking Claude to review code:
125+
126+
```console
127+
128+
Please review this [test/function/module] for:
129+
130+
- Compliance with project standards (CLAUDE.md, code_standards.md).
131+
- Correct usage of the `pre` fixture.
132+
- Proper error handling and type annotations.
133+
- Performance considerations.
134+
135+
**Code**:
136+
137+
```python
138+
[your code here]
139+
```console
140+
141+
**Specific concerns**: [any particular areas you're unsure about]
142+
143+
## 🧠 Understanding LLM Limitations
144+
145+
### What Claude Excels At
146+
147+
- ✅ **Code patterns and structure**.
148+
- ✅ **Following established conventions**.
149+
- ✅ **Debugging common issues**.
150+
- ✅ **Explaining complex concepts**.
151+
- ✅ **Generating boilerplate code**.
152+
- ✅ **Reviewing code for standards compliance**.
153+
154+
### What to Verify Independently
155+
156+
- ⚠️ **Latest dependency versions** (always check official docs).
157+
- ⚠️ **New EIP specifications** (verify against ethereum/EIPs).
158+
- ⚠️ **Breaking changes** in recent updates.
159+
- ⚠️ **Environment-specific issues** (OS, architecture).
160+
- ⚠️ **Security implications** of suggestions.
161+
162+
### For Effective LLM Collaboration
163+
164+
- Provide clear, specific prompts.
165+
- Break complex tasks into smaller pieces.
166+
- Always validate LLM output against standards.
167+
- Use the codebase's existing patterns as examples.
168+
169+
Remember: **LLMs are powerful tools that work best when given good context and clear objectives.** The better you understand this codebase, the better you can guide Claude to help you effectively.

‎docs/CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ Users can select any of the artifacts depending on their testing needs for their
5252
- ✨ Add documentation "Running Tests" that explains the different methods available to run EEST tests and reference guides for running `consume` and `hive`: ([#1172](https://github.com/ethereum/execution-spec-tests/pull/1172)).
5353
- ✨ Added a new `eest` sub-command, `eest info`, to easily print a cloned EEST repository's version and the versions of relevant tools, e.g., `python`, `uv` ([#1621](https://github.com/ethereum/execution-spec-tests/pull/1621)).
5454
- ✨ Add `CONTRIBUTING.md` for execution-spec-tests and improve coding standards documentation ([#1604](https://github.com/ethereum/execution-spec-tests/pull/1604)).
55+
- ✨ Add `CLAUDE.md` to help working in @ethereum/execution-spec-tests with [Claude Code](https://docs.anthropic.com/en/docs/claude-code/overview) ([#1749](https://github.com/ethereum/execution-spec-tests/pull/1749)).
5556
- ✨ Use `codespell` instead of `pyspelling` to spell-check python and markdown sources ([#1715](https://github.com/ethereum/execution-spec-tests/pull/1715)).
5657
- 🔀 Updated from pytest 7 to [pytest 8](https://docs.pytest.org/en/stable/changelog.html#features-and-improvements), benefits include improved type hinting and hook typing, stricter mark handling, and clearer error messages for plugin and metadata development ([#1433](https://github.com/ethereum/execution-spec-tests/pull/1433)).
5758
- 🐞 Fix bug in ported-from plugin and coverage script that made PRs fail with modified tests that contained no ported tests ([#1661](https://github.com/ethereum/execution-spec-tests/pull/1661)).

0 commit comments

Comments
 (0)