Skip to content

Commit f1f7af6

Browse files
marioevzCarsons-Eels
marioevz
authored and
Carsons-Eels
committed
feat(clis): extract_config command (ethereum#1740)
1 parent 4065238 commit f1f7af6

File tree

5 files changed

+434
-0
lines changed

5 files changed

+434
-0
lines changed

‎docs/CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ Users can select any of the artifacts depending on their testing needs for their
5757
- 🐞 Fix bug in ported-from plugin and coverage script that made PRs fail with modified tests that contained no ported tests ([#1661](https://github.com/ethereum/execution-spec-tests/pull/1661)).
5858
- 🔀 Refactor the `click`-based CLI interface used for pytest-based commands (`fill`, `execute`, `consume`) to make them more extensible ([#1654](https://github.com/ethereum/execution-spec-tests/pull/1654)).
5959
- 🔀 Split `src/ethereum_test_types/types.py` into several files to improve code organization ([#1665](https://github.com/ethereum/execution-spec-tests/pull/1665)).
60+
- ✨ Added `extract_config` command to extract genesis files used to launch clients in hive ([#1740](https://github.com/ethereum/execution-spec-tests/pull/1740)).
6061
- ✨ Added automatic checklist generation for every EIP inside of the `tests` folder. The checklist is appended to each EIP in the documentation in the "Test Case Reference" section ([#1679](https://github.com/ethereum/execution-spec-tests/pull/1679), [#1718](https://github.com/ethereum/execution-spec-tests/pull/1718)).
6162

6263
### 🧪 Test Cases

‎docs/library/cli/extract_config.md

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# `extract_config` - Extract Client Configuration Files
2+
3+
The `extract_config` command extracts configuration files from Ethereum clients by spawning them via Hive and retrieving the generated files from the Docker container.
4+
5+
## Purpose
6+
7+
When Ethereum clients start up with a genesis configuration, they generate various configuration files such as:
8+
9+
- `/chainspec/test.json` - Chain specification file
10+
- `/configs/test.cfg` - Configuration file
11+
- `/genesis.json` - Genesis block configuration
12+
13+
This tool automates the process of extracting these files from the client containers for analysis or debugging purposes.
14+
15+
## Usage
16+
17+
```bash
18+
uv run extract_config --fixture <FIXTURE_PATH> [OPTIONS]
19+
```
20+
21+
### Options
22+
23+
- `--fixture, -f` (required): Path to a fixture JSON file or directory containing fixture files
24+
- `--client, -c`: Specific client name to extract from (e.g., go-ethereum, besu, nethermind). If not specified, extracts from all available clients
25+
- `--output, -o`: Output directory for extracted files (default: ./extracted_configs)
26+
- `--hive-url`: Hive server URL (default: http://127.0.0.1:3000)
27+
- `--list-files, -l`: List files in the container root before extraction
28+
- `--help`: Show help message
29+
30+
### Examples
31+
32+
Extract configuration from all clients using a specific fixture:
33+
34+
```bash
35+
uv run extract_config --fixture fixtures/blockchain_tests/paris/security/test_selfdestruct_balance_bug.json
36+
```
37+
38+
Extract configuration from a specific client:
39+
40+
```bash
41+
uv run extract_config --fixture fixtures/blockchain_tests/paris/security/test_selfdestruct_balance_bug.json --client besu
42+
```
43+
44+
Extract configurations from all fixtures in a directory:
45+
46+
```bash
47+
uv run extract_config --fixture fixtures/blockchain_tests/paris/security/
48+
```
49+
50+
Extract to a specific directory and list container files:
51+
52+
```bash
53+
uv run extract_config --fixture my_fixture.json --output ./my_configs --list-files
54+
```
55+
56+
## Prerequisites
57+
58+
1. Hive must be running in the background:
59+
60+
```bash
61+
./hive --dev
62+
```
63+
64+
2. Docker must be installed and accessible
65+
66+
## Output
67+
68+
The tool creates a hierarchical directory structure:
69+
70+
```console
71+
<output_dir>/
72+
<fixture_name>/
73+
<client_name>/
74+
chainspec.json
75+
config.cfg
76+
genesis.json
77+
```
78+
79+
For example:
80+
81+
```console
82+
extracted_configs/
83+
test_selfdestruct_balance_bug/
84+
go-ethereum/
85+
genesis.json
86+
besu/
87+
genesis.json
88+
chainspec.json
89+
nethermind/
90+
chainspec.json
91+
config.cfg
92+
```
93+
94+
Only files that exist in the client container will be extracted.
95+
96+
## How It Works
97+
98+
1. Loads the fixture file(s) to extract genesis configuration
99+
2. Starts a Hive simulation
100+
3. For each fixture and each client:
101+
- Captures the list of Docker containers before starting the client
102+
- Spawns the client with the genesis configuration
103+
- Compares Docker containers to identify the newly created container
104+
- Uses Docker exec commands to check for and extract configuration files
105+
- Saves the extracted files to the organized output directory
106+
- Stops the client container
107+
4. Ends the Hive simulation
108+
109+
## Container ID Detection
110+
111+
Since Hive doesn't directly expose container IDs, the tool uses a detection mechanism:
112+
113+
1. Lists all Docker container IDs before starting the client
114+
2. Starts the client through Hive
115+
3. Lists all Docker container IDs after starting the client
116+
4. The difference should be exactly one container - the client's container
117+
118+
## Supported Fixture Formats
119+
120+
The tool supports:
121+
122+
- Individual fixture JSON files (BlockchainFixture format)
123+
- SharedPreStateGroup JSON files
124+
- Directories containing multiple fixture files
125+
126+
## Troubleshooting
127+
128+
- If no files are extracted, use the `--list-files` flag to see what files are available in the container root
129+
- Ensure Hive is running before executing the command
130+
- Check that Docker is installed and the current user has permissions to run Docker commands
131+
- If the tool fails to detect the container ID, ensure no other containers are being created simultaneously
132+
- Some clients may not generate all configuration file types - this is normal

‎docs/library/cli/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,3 +3,4 @@
33
- [`check_eip_versions`](../../writing_tests/reference_specification.md) - A CLI tool to check the SHA values specified in EIP tests match the latest version from ethereum/EIPs.
44
- [`eest`](eest.md) - A CLI tool that helps with routine tasks in ethereum/execution-spec-tests.
55
- [`evm_bytes`](evm_bytes.md) - Convert the given EVM bytes from a binary file or a hex string to EEST's python opcodes.
6+
- [`extract_config`](extract_config.md) - Extract client configuration files (chainspec/genesis.json) from Ethereum clients via Hive.

‎pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,7 @@ hasher = "cli.hasher:main"
103103
eest = "cli.eest.cli:eest"
104104
fillerconvert = "cli.fillerconvert.fillerconvert:main"
105105
groupstats = "cli.show_pre_alloc_group_stats:main"
106+
extract_config = "cli.extract_config:extract_config"
106107

107108
[tool.setuptools.packages.find]
108109
where = ["src"]

0 commit comments

Comments
 (0)