Skip to content

feat: config docs generator #6171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Jun 19, 2025
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
a6f68bd
Add config-docs-generator tool for automated documentation generation
Jiloc Jun 3, 2025
5d74eee
move Dockerfile to the right folder
Jiloc Jun 3, 2025
5f353cb
clippy and fmt
Jiloc Jun 3, 2025
0ad7569
add support for @required and @units annotations. Implement parsing f…
Jiloc Jun 4, 2025
e5b6f92
add documentation of format
Jiloc Jun 4, 2025
dcef804
improve html formatting
Jiloc Jun 4, 2025
d2a0bfa
Merge branch 'chore/enhance-config-docs' into feat/config-docs-generator
Jiloc Jun 4, 2025
d5084f9
update configuration-reference.md
Jiloc Jun 4, 2025
4aeb7c1
Merge branch 'chore/enhance-config-docs' into feat/config-docs-generator
Jiloc Jun 4, 2025
fe86039
update reference
Jiloc Jun 4, 2025
a2b1508
add custom section mappings and template support
Jiloc Jun 6, 2025
51c0e10
fix tests
Jiloc Jun 6, 2025
cb07367
Merge branch 'chore/enhance-config-docs' into feat/config-docs-generator
Jiloc Jun 10, 2025
90de815
update generated docs
Jiloc Jun 10, 2025
31beefe
remove CARGO_HOME from dockerfile
Jiloc Jun 10, 2025
c2212a2
simplify json parsing logic
Jiloc Jun 10, 2025
45917e4
improve test coverage
Jiloc Jun 10, 2025
d40be62
make required parsing strict. only true/false
Jiloc Jun 10, 2025
39ddc9c
use mapping file for TARGET_STRUCTS
Jiloc Jun 12, 2025
01005ce
Merge branch 'develop' into feat/config-docs-generator
Jiloc Jun 13, 2025
5e5e6a2
do not run docker as root
Jiloc Jun 16, 2025
358fe35
remove generated configuration documentation
Jiloc Jun 16, 2025
e36db35
Merge branch 'develop' into feat/config-docs-generator
Jiloc Jun 16, 2025
0366c26
add prerequisites for running locally
Jiloc Jun 17, 2025
826c7cc
generate docs in target folder
Jiloc Jun 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
238 changes: 237 additions & 1 deletion contrib/tools/config-docs-generator/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Configuration Documentation Generator

A tool that automatically generates comprehensive Markdown documentation for Stacks node TOML configuration options. The documentation is extracted directly from Rust source code comments and generates a complete configuration reference.
This tool automatically generates markdown documentation from Rust configuration structs by extracting specially formatted doc comments.

## Quick Start

Expand Down Expand Up @@ -54,6 +54,242 @@ For each configuration field, it extracts:
- **Primary**: `docs/generated/configuration-reference.md` - Complete configuration reference
- **Intermediate**: `target/doc-generation/extracted-config-docs.json` - Raw extracted data

## Annotation Syntax Guide

### Overview

The generator processes doc comments with a structured annotation format:

```rust
/// [Description text in Markdown format]
/// ---
/// @annotation_name: value
/// @another_annotation: value
pub field_name: Type,
```

### General Structure

- **Description**: Standard Markdown text before the `---` separator
- **Separator**: Three dashes (`---`) separate description from annotations
- **Annotations**: Key-value pairs starting with `@`, each on its own line

### Supported Annotations

#### `@default: <value>`
Specifies the default value for the field.
- **Value Type**: String
- **Multiline Support**: Yes (all modes)
- **Examples**:
```rust
/// @default: `None`
/// @default: `"localhost:8080"`
/// @default: |
/// Complex multi-line
/// default value
```

#### `@notes: <content>`
Additional notes or explanations, rendered as a bulleted list.
- **Value Type**: String (parsed into list items)
- **Multiline Support**: Yes (all modes)
- **List Processing**: Lines starting with `-`, `*`, or `•` become list items
- **Examples**:
```rust
/// @notes: Single line note
/// @notes:
/// - First bullet point
/// - Second bullet point
/// @notes: |
/// Complex formatting with
/// preserved line breaks
```

#### `@deprecated: <message>`
Marks a field as deprecated with an optional message.
- **Value Type**: String
- **Multiline Support**: Yes (all modes)
- **Examples**:
```rust
/// @deprecated: Use new_field instead
/// @deprecated: |
/// This field will be removed in v3.0.
/// Migrate to the new configuration system.
```

#### `@toml_example: <example>`
Provides TOML configuration examples.
- **Value Type**: String
- **Multiline Support**: Yes (all modes)
- **Rendering**: Displayed in `<pre><code>` blocks in markdown tables
- **Examples**:
```rust
/// @toml_example: key = "value"
/// @toml_example: |
/// [section]
/// key = "value"
/// nested = { a = 1, b = 2 }
```

#### `@required: <boolean>`
Indicates whether the field is mandatory.
- **Value Type**: Boolean (flexible parsing)
- **Default**: `false` if annotation is omitted
- **Supported Values**:
- `true`, `True`, `TRUE`, `yes`, `Yes`, `YES`, `1` → `true`
- `false`, `False`, `FALSE`, `no`, `No`, `NO`, `0` → `false`
- Invalid values default to `false`
- **Examples**:
```rust
/// @required: true
/// @required: yes
/// @required: false
```

#### `@units: <unit>`
Specifies the unit of measurement for the field.
- **Value Type**: String
- **Multiline Support**: Yes (all modes)
- **Constant References**: Supports `[`CONSTANT_NAME`]` syntax
- **Examples**:
```rust
/// @units: milliseconds
/// @units: sats/vByte
```

### Multiline Content Support

All annotations support three multiline modes:

#### Default Literal-like Mode
Content preserves newlines and relative indentation within the annotation block.

```rust
/// @notes:
/// First line with base indentation
/// Second line more indented
/// Third line back to base
/// Fourth line very indented
```

**Output preserves relative indentation**:
```
First line with base indentation
Second line more indented
Third line back to base
Fourth line very indented
```

#### Literal Block Style (`|`)
Exact preservation of newlines and relative indentation. Uses "clip" chomping (single trailing newline preserved).

```rust
/// @toml_example: |
/// [network]
/// bind = "0.0.0.0:20444"
/// # Indented comment
/// timeout = 30
```

**Output**:
```
[network]
bind = "0.0.0.0:20444"
# Indented comment
timeout = 30
```

#### Folded Block Style (`>`)
Folds lines into paragraphs with intelligent spacing. More-indented lines preserved as literal blocks.

```rust
/// @notes: >
/// This is a long paragraph that will be
/// folded into a single line with spaces
/// between the original line breaks.
///
/// This is a second paragraph after a blank line.
///
/// This indented block will be preserved
/// exactly as written, like code.
///
/// Back to normal folded paragraph text.
```

**Output**:
```
This is a long paragraph that will be folded into a single line with spaces between the original line breaks.

This is a second paragraph after a blank line.

This indented block will be preserved
exactly as written, like code.

Back to normal folded paragraph text.
```

### Same-line Content

Content can start immediately after the colon for default multiline mode:

```rust
/// @default: immediate content
/// @notes: Content that starts immediately
/// and continues on the next line
```

For literal (`|`) and folded (`>`) modes, content must start on the next line:

```rust
/// @notes: |
/// Content starts here on the next line
/// All content must be indented on subsequent lines
/// @deprecated: >
/// Folded content also starts on the next line
/// and will be joined appropriately
```

### Complete Example

```rust
/// Timeout duration for network connections.
///
/// This setting controls how long the node will wait for network operations
/// to complete before timing out. Setting this too low may cause connection
/// failures on slow networks.
/// ---
/// @default: [`DEFAULT_NETWORK_TIMEOUT`]
/// @required: true
/// @units: milliseconds
/// @notes:
/// - Must be greater than 0
/// - Recommended range: 1000-30000
/// - Higher values needed for slow connections
/// @toml_example: |
/// [network]
/// timeout = 15000 # 15 seconds
/// @deprecated: >
/// Use the new `connection_timeout` setting instead.
/// This field will be removed in version 3.0.
pub timeout_ms: u64,
```

### Best Practices

1. **Choose the right multiline mode**:
- Default mode: General text with preserved formatting
- Literal (`|`): Code examples, exact formatting required
- Folded (`>`): Documentation prose, automatic paragraph wrapping

2. **Use constant references in `@default` when appropriate**

### Integration with Rust Documentation

This system integrates with standard Rust documentation tools:
- Doc comments remain valid for `rustdoc`
- Annotations are ignored by standard documentation generators
- Full compatibility with existing documentation workflows

## Adding New Configuration Structs

### 1. Update the Target List
Expand Down
Loading