Improve support for doctests

[taiki-e]

STATUS:

#40 fixed all of the errors and many of the warnings

The remaining warnings are probably the column offset errors mentioned in rust-lang/rust#79417 (comment), and it's not clear whether we can fix them on our side at this time.

---

This is currently available via --doctests flag as an unstable feature.

cargo llvm-cov --doctests

Refs:

• https://doc.rust-lang.org/rustc/instrument-coverage.html#including-doc-tests
• Rustdoc doctests should have their file/lines remapped when using -Zinstrument-coverage rust-lang/rust#79417
• Tracking issue: --persist-doctests flag for rustdoc rust-lang/rust#56925

[taiki-e]

STATUS: I think it's better than before, but there are still a few problems.

warning: 4 functions have mismatched data

[cloudhead]

It would be great to have this working better. This tool is super handy.

Currently, running with --doctests I get some error output at the end, and it seems like the files that contain doctests get included twice in the coverage output:

error: src/block.rs: No such file or directory
warning: The file 'src/block.rs' isn't covered.
error: src/block/filter.rs: No such file or directory
warning: The file 'src/block/filter.rs' isn't covered.
error: src/block/iter.rs: No such file or directory
warning: The file 'src/block/iter.rs' isn't covered.
error: src/network.rs: No such file or directory
warning: The file 'src/network.rs' isn't covered.
error: src/protocol/addrmgr.rs: No such file or directory
warning: The file 'src/protocol/addrmgr.rs' isn't covered.
error: src/time.rs: No such file or directory
warning: The file 'src/time.rs' isn't covered.

[taiki-e]

@cloudhead Is there an example that can reproduce the error you encountered?

[cloudhead]

Yeah, though after reproducing this I realize some of these errors come from the --html flag, and on its own, --doctests doesn't output these errors, only in combination with --html.

The repo is here:
https://github.com/cloudhead/nakamoto

I ran: cargo llvm-cov --all --doctests --html

The issue with coverage output for files included twice appears with only --doctests though. Basically the paths are not matched, eg.:

net/poll/src/time.rs                     45                22    51.11%          14                 5    64.29%          78                30    61.54%           0                 0         -
src/time.rs                              22                 2    90.91%           4                 0   100.00%          89                 0   100.00%           0                 0         -

These are actually the same file, but I'm guessing the second one is from the doctest run, while the first is from the unit test run, I'm not sure.

Note that there is no src/time.rs, that path doesn't exist.

[taiki-e]

Thanks. I'll take a look.

[taiki-e]

#40 fixed all of the errors and many of the warnings that happen in cloudhead/nakamoto.

Before #40:

warning: 5 functions have mismatched data
error: src/block.rs: No such file or directory
warning: The file 'src/block.rs' isn't covered.
error: src/block/filter.rs: No such file or directory
warning: The file 'src/block/filter.rs' isn't covered.
error: src/block/iter.rs: No such file or directory
warning: The file 'src/block/iter.rs' isn't covered.
error: src/network.rs: No such file or directory
warning: The file 'src/network.rs' isn't covered.
error: src/protocol/addrmgr.rs: No such file or directory
warning: The file 'src/protocol/addrmgr.rs' isn't covered.
error: src/time.rs: No such file or directory
warning: The file 'src/time.rs' isn't covered.
warning: 5 functions have mismatched data

After #40:

warning: 5 functions have mismatched data
warning: 5 functions have mismatched data

The remaining warnings are probably the column offset errors mentioned in rust-lang/rust#79417 (comment), and it's not clear whether we can fix them on our side at this time.

[cloudhead]

Awesome, thanks a bunch!