feat(docs): auto-generate a page showing fill's command-line options (ethereum#1747)

danceratopz · Carsons-Eels · commit 40652384e47a · 2025-07-01T05:39:15.000-04:00
diff --git a/docs/filling_tests/filling_tests_command_line.md b/docs/filling_tests/filling_tests_command_line.md
@@ -111,78 +111,4 @@ To list the options that only specific to fill, use:
 fill --help
 ```
 
-Output:
-
-```text
-usage: fill [-h] [--strict-alloc] [--ca-start CA_START] [--ca-incr CA_INCR]
-            [--evm-code-type EVM_CODE_TYPE] [--solc-bin SOLC_BIN] [--solc-version SOLC_VERSION]
-            [--evm-bin EVM_BIN] [--traces] [--verify-fixtures]
-            [--verify-fixtures-bin VERIFY_FIXTURES_BIN] [--filler-path FILLER_PATH] [--output OUTPUT]
-            [--flat-output] [--single-fixture-per-file] [--no-html] [--build-name BUILD_NAME]
-            [--skip-index SKIP_INDEX] [--block-gas-limit BLOCK_GAS_LIMIT] [--evm-dump-dir EVM_DUMP_DIR]
-            [--skip-evm-dump] [--forks] [--fork FORK] [--from FROM] [--until UNTIL]
-
-options:
-  -h, --help            show this help message and exit
-
-Arguments defining pre-allocation behavior during test filling.:
-  --strict-alloc        [DEBUG ONLY] Disallows deploying a contract in a predefined address.
-  --ca-start, --contract-address-start CA_START
-                        The starting address from which tests will deploy contracts.
-  --ca-incr, --contract-address-increment CA_INCR
-                        The address increment value to each deployed contract by a test.
-  --evm-code-type EVM_CODE_TYPE
-                        Type of EVM code to deploy in each test by default.
-
-Arguments defining the solc executable:
-  --solc-bin SOLC_BIN   Path to a solc executable (for Yul source compilation). No default; if
-                        unspecified `--solc-version` is used.
-  --solc-version SOLC_VERSION
-                        Version of the solc compiler to use. Default: 0.8.24.
-
-Arguments defining evm executable behavior:
-  --evm-bin EVM_BIN     Path to an evm executable (or name of an executable in the PATH) that provides
-                        `t8n`. Default: `ethereum-spec-evm-resolver`.
-  --traces              Collect traces of the execution information from the transition tool.
-  --verify-fixtures     Verify generated fixture JSON files using geth's evm blocktest command. By
-                        default, the same evm binary as for the t8n tool is used. A different (geth) evm
-                        binary may be specified via --verify-fixtures-bin, this must be specified if
-                        filling with a non-geth t8n tool that does not support blocktest.
-  --verify-fixtures-bin VERIFY_FIXTURES_BIN
-                        Path to an evm executable that provides the `blocktest` command. Default: The
-                        first (geth) 'evm' entry in PATH.
-
-Arguments defining filler location and output:
-  --filler-path FILLER_PATH
-                        Path to filler directives
-  --output OUTPUT       Directory path to store the generated test fixtures. If the specified path ends
-                        in '.tar.gz', then the specified tarball is additionally created (the fixtures
-                        are still written to the specified path without the '.tar.gz' suffix). Can be
-                        deleted. Default: './fixtures'.
-  --flat-output         Output each test case in the directory without the folder structure.
-  --single-fixture-per-file
-                        Don't group fixtures in JSON files by test function; write each fixture to its
-                        own file. This can be used to increase the granularity of --verify-fixtures.
-  --no-html             Don't generate an HTML test report (in the output directory). The --html flag can
-                        be used to specify a different path.
-  --build-name BUILD_NAME
-                        Specify a build name for the fixtures.ini file, e.g., 'stable'.
-  --skip-index SKIP_INDEX
-                        Skip generating an index file for all produced fixtures.
-  --block-gas-limit BLOCK_GAS_LIMIT
-                        Default gas limit used ceiling used for blocks and tests that attempt to consume
-                        an entire block's gas. (Default: 72000000)
-
-Arguments defining debug behavior:
-  --evm-dump-dir, --t8n-dump-dir EVM_DUMP_DIR
-                        Path to dump the transition tool debug output. (Default:
-                        <repo>/logs/evm)
-  --skip-evm-dump, --skip-t8n-dump
-                        Skip dumping the the transition tool debug output.
-
-Specify the fork range to generate fixtures for:
-  --forks               Display forks supported by the test framework and exit.
-  --fork FORK           Only fill tests for the specified fork.
-  --from FROM           Fill tests from and including the specified fork.
-  --until UNTIL         Fill tests until and including the specified fork.
-```
+For a complete, up-to-date list of all command-line options, see the [Fill Command-Line Options](filling_tests_command_line_options.md) page, which is automatically generated from the current `fill --help` output.
diff --git a/docs/navigation.md b/docs/navigation.md
@@ -29,6 +29,7 @@
   * [Filling Tests](filling_tests/index.md)
       * [Getting Started](filling_tests/getting_started.md)
       * [Filling Tests at a Prompt](filling_tests/filling_tests_command_line.md)
+      * [Fill Command-Line Options](filling_tests/filling_tests_command_line_options.md)
       * [Filling Tests in VS Code](filling_tests/filling_tests_vs_code.md)
       * [Filling Tests for Features Under Development](filling_tests/filling_tests_dev_fork.md)
       * [An Explanation of Test IDs](filling_tests/test_ids.md)
diff --git a/docs/scripts/generate_fill_help.py b/docs/scripts/generate_fill_help.py
@@ -0,0 +1,114 @@
+#!/usr/bin/env python3
+"""
+Generate the fill command help output for documentation.
+
+This script captures the output of 'fill --help' and generates a complete
+documentation page that includes both static content and the auto-generated
+help output. The generated page replaces the manual help output with
+current command-line options.
+"""
+
+import logging
+import subprocess
+import sys
+import textwrap
+from pathlib import Path
+
+import mkdocs_gen_files
+
+logger = logging.getLogger("mkdocs")
+
+
+def get_fill_help_output() -> str:
+    """Run 'fill --help' and capture its output."""
+    # Get the repository root (assuming script is in docs/scripts/)
+    script_dir = Path(__file__).parent
+    repo_root = script_dir.parent.parent
+
+    try:
+        result = subprocess.run(
+            ["uv", "run", "fill", "--help"],
+            capture_output=True,
+            text=True,
+            check=True,
+            cwd=repo_root,  # Run from repo root where pytest.ini exists
+        )
+        return result.stdout
+    except subprocess.CalledProcessError as e:
+        logger.error(f"Error running 'fill --help': {e}")
+        logger.error(f"stderr: {e.stderr}")
+        sys.exit(1)
+
+
+def format_help_output(help_text: str, max_width: int = 88) -> str:
+    """
+    Format the help output with proper line wrapping.
+
+    Args:
+        help_text: The raw help output
+        max_width: Maximum line width (default 88 to match existing docs)
+
+    Returns:
+        Formatted help text suitable for documentation
+
+    """
+    lines = help_text.splitlines()
+    formatted_lines = []
+
+    for line in lines:
+        # Don't wrap lines that are part of the usage section or are empty
+        if not line.strip() or line.startswith("usage:") or line.startswith("  ") and "--" in line:
+            formatted_lines.append(line)
+        else:
+            # Wrap long lines while preserving indentation
+            indent = len(line) - len(line.lstrip())
+            if len(line) > max_width and indent == 0:
+                wrapped = textwrap.fill(
+                    line,
+                    width=max_width,
+                    subsequent_indent="  ",
+                    break_long_words=False,
+                    break_on_hyphens=False,
+                )
+                formatted_lines.append(wrapped)
+            else:
+                formatted_lines.append(line)
+
+    return "\n".join(formatted_lines)
+
+
+def generate_command_line_options_docs():
+    """Generate a standalone page with just the fill command-line options."""
+    # Get and format the help output
+    help_output = get_fill_help_output()
+    formatted_output = format_help_output(help_output)
+
+    # Create the complete page content
+    page_content = f"""# Fill Command-Line Options
+
+Fill is a [pytest](https://docs.pytest.org/en/stable/)-based command. This page lists custom
+options that the `fill` command provides. To see the full list of options that is available
+to fill (including the standard pytest and plugin command-line options) use `fill --pytest-help`.
+
+*This page is automatically generated from the current `fill --help` output.*
+
+## Command Help Output
+
+```text
+{formatted_output}
+```
+
+---
+
+*This page was automatically generated from `fill --help` output.*
+"""
+
+    # Write the generated content to a virtual file
+    with mkdocs_gen_files.open("filling_tests/filling_tests_command_line_options.md", "w") as f:
+        f.write(page_content)
+
+    logger.info("Generated filling_tests_command_line_options.md with current fill --help output")
+
+
+# Run the generation
+generate_command_line_options_docs()
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -28,6 +28,7 @@ plugins:
       scripts:
         - docs/scripts/copy_repo_docs_to_mkdocs.py
         - docs/scripts/gen_test_case_reference.py
+        - docs/scripts/generate_fill_help.py
   - literate-nav:
       nav_file: navigation.md
 
