Documentation inheritance issue with included extension modules

[johnhaley81]

Documentation inheritance issue with included extension modules

Problem Description

Functions defined in extension modules (like FoldableExtensions) lose their documentation when included in user-facing modules like List and Array. This creates a significant gap in the generated API documentation where core utility functions appear completely undocumented.

Concrete Example

The all function is defined in src/extensions/Relude_Extensions_Foldable.re with comprehensive documentation:

/**
Indicates if all items in the foldable satisfy the given predicate.
*/
let all: 'a. ('a => bool, F.t('a)) => bool =
  (f, xs) => F.fold_left((v, x) => v && f(x), true, xs);

However, when users access this via List.all or Array.all (through the include statement in *_Instances.re files), the generated HTML documentation shows the function signature but no documentation text.

Affected Functions

All functions from FoldableExtensions that users interact with through collection modules:

• all, any - predicate checking
• contains, containsBy - membership testing
• length, size, count - counting
• find, findWithIndex - searching
• forEach, forEachWithIndex - iteration
• min, max, minBy, maxBy - extrema
• toList, toArray - conversions
• Plus functions from other extension modules (MonoidExtensions, EqExtensions, etc.)

Estimated impact: ~20+ core utility functions per collection type appear undocumented.

How to Verify the Issue

1. Build documentation: make docs
2. Open _build/default/_doc/_html/relude/Relude/List/index.html
3. Search for id="val-all" - you'll find the function signature but no <div class="spec-doc"> section
4. Compare with _build/default/_doc/_html/relude/Relude/Extensions/Foldable/FoldableExtensions/index.html where the documentation is complete

Root Cause Analysis

This is a fundamental limitation of ReasonML/OCaml's module system:

1. Module functors like FoldableExtensions(F) define documentation at the functor level
2. Include statements (include Relude_Extensions_Foldable.FoldableExtensions(Foldable)) copy function implementations but not odoc comments
3. Generated documentation only sees the final module signature, not the source of included functions
4. IDE tooling and Context7-style documentation crawlers see functions without their original documentation

Impact on Context7 Integration

This directly affects issue #353's goal of Context7 integration because:

• Context7 crawls the generated documentation (HTML/markdown)
• Missing documentation means core utility functions appear as undocumented in AI/LLM contexts
• Users get incomplete API information when using Claude Code or similar tools
• Significantly reduces the value of Relude in AI-assisted development

Potential Solutions

Option 1: Manual Documentation Duplication

Approach: Copy documentation comments to each inclusion site
Pros: Guaranteed to work with current tooling
Cons: Maintenance burden, documentation drift risk, violates DRY principle

Option 2: Preprocessing Script Solution

Approach: Build-time script that copies documentation from extension modules to included locations
Pros: Automated, maintains single source of truth
Cons: Complex build process, needs custom tooling

Option 3: odoc Attribute Workarounds

Approach: Use [@@@ocaml.doc] attributes or similar odoc directives
Pros: Works within existing system
Cons: Limited effectiveness for included modules, still experimental

Option 4: Upstream Tooling Improvements

Approach: Contribute to odoc to support documentation inheritance
Pros: Benefits entire OCaml ecosystem
Cons: Long-term solution, no immediate fix

Recommended Approach for Context7

For the Context7 integration effort, I recommend Option 2 (preprocessing script) because:

1. Immediate fix: Can be implemented as part of the documentation generation pipeline
2. Automation: No ongoing maintenance burden once implemented
3. Context7 compatibility: Generates properly documented HTML/markdown that Context7 can crawl
4. Reversible: Can be removed if upstream solutions emerge

Implementation Plan

1. Create scripts/copy-extension-docs.js that:
   • Parses extension module .re files for documentation comments
   • Identifies where those functions are included in collection modules
   • Copies documentation to a temporary location before odoc generation
2. Integrate into documentation generation workflow (make docs)
3. Test with Context7 submission to verify improved documentation quality

Success Criteria

• All ~20+ affected functions show proper documentation in generated HTML
• Context7 can successfully index comprehensive Relude API documentation
• Zero documentation gaps for core utility functions accessed through List/Array/etc.
• Automated solution requires no manual maintenance

---

This issue represents a significant barrier to AI/LLM accessibility of the Relude library and should be prioritized for the Context7 integration effort.

[mlms13]

There's a technique that might help that I've never actually tried:

• Move the doc comments in the e.g. FoldableExtensions file to an interface (e.g. rei or mli)
• Add an interface for e.g. List
• Make the List interface include the interface from FoldableExtensions

I know there's been mixed feelings in the past about adding interface files, but I think tooling in OCaml (particularly around docs) might be happier if it finds docs in the interface files. No idea if this will actually resolve the issue, though.

---

For the script-based approach to copy docs between modules, I think we could use that to make some progress, but I'm worried that it would require a lot of copying and it wouldn't hold up well as we add new data types, new typeclass helpers, etc. I think we probably have quite a few more than 20 affected functions at this point. e.g. most things in Option would need to copy all the docs from Functor and Applicative and Monad and Foldable and MonoidAny and SemigroupAny, and then the same is true not just for Option but also for Result and List and Array and IO and probably primitive types like String and Int and Float. The number of combinations is pretty huge.

[johnhaley81]

@mlms13 Thanks for the thoughtful suggestions! I've done extensive research into this issue and want to share my findings.

Interface Files Approach

I investigated the .rei approach you suggested. Unfortunately, after testing, I found that interface files don't solve the documentation inheritance problem for our use case. When you include a module interface that was created from a functor application, odoc still doesn't pull the documentation through. The fundamental issue is that odoc doesn't trace documentation through functor applications, whether in .re or .rei files.

Additionally, as you noted, Relude's extensive use of include statements makes .rei files particularly challenging to maintain. The module composition patterns that give Relude its power would require constant synchronization between implementation and interface files.

Research into Native odoc Solutions

I conducted thorough research into native odoc capabilities:

• No inheritance directives exist (@inherit, @include, @copy don't exist)
• No functor documentation inheritance is supported
• Alternative tools (MDX, Voodoo, Sherlodoc) don't solve this problem
• Major OCaml projects (like Jane Street's Base) handle this through code generation rather than documentation inheritance

Current Implementation Update

Good news: I've successfully implemented and tested the script-based approach! Here's what's working:

✅ Automated extraction of all documented functions from extension modules
✅ Dynamic injection of comprehensive documentation blocks before include statements
✅ Build integration via Makefile (make docs handles everything automatically)
✅ Verified working - added a test function foo to Foldable and confirmed it appears in all target modules' documentation

The script currently handles 20 Foldable functions across 7 target modules successfully.

Addressing Scalability Concerns

Your scalability concerns are well-founded. The current implementation has:

• Hardcoded function lists (fragile)
• Manual module mapping (doesn't scale)
• Limited to Foldable extensions only

Proposed enhancements to address these:

1. Auto-discovery: Parse extension modules dynamically to find all documented functions automatically
2. Pattern detection: Automatically identify which modules include which extensions
3. Comprehensive coverage: Expand to all extension types (Apply, Functor, Monad, Alt, Plus, etc.)
4. Robust parsing: Replace regex with proper AST parsing for reliability

With these enhancements, the solution would:

• Automatically handle new functions added to extensions
• Automatically detect new modules that include extensions
• Scale to any number of extension types
• Require zero maintenance as the library grows

Why This Approach is Our Best Option

After extensive research, I believe the enhanced script approach is our most pragmatic solution because:

1. It works - We have verified proof with actual documentation appearing in HTML output
2. It's automatic - Once enhanced, it requires no manual maintenance
3. Single source of truth - Documentation stays in extension modules
4. Build-integrated - Seamless part of make docs workflow
5. Future-compatible - Easy to remove if odoc ever adds native support

The alternative would be manually duplicating hundreds of function documentations across dozens of modules, which would be a maintenance nightmare and prone to drift.

Next Steps

I'm ready to implement the enhancements to make this solution robust and scalable. The plan is:

1. Replace hardcoded lists with dynamic discovery
2. Improve pattern matching for reliability
3. Expand to all extension types
4. Add comprehensive testing

This would give us a zero-maintenance solution that automatically ensures all extension functions are properly documented, solving the core issue while we wait for potential future odoc improvements.

What do you think? Should I proceed with these enhancements?