Describe the issue

Coverage score is affected by non-library code (test code and testenv code), also, branch coverage is not exercised:

Commands & Reports

grcov . --binary-path ./target/debug/ -s . -t html --branch --ignore-not-existing --keep-only '**/crates/**' --ignore '**/tests/**' --ignore '**/examples/**' -o target/coverage/html

grcov . --binary-path ./target/debug/ -s . -t html --branch --ignore-not-existing --keep-only '**/crates/**' --ignore '**/crates/testenv/**' --ignore '**/tests/**' --ignore '**/examples/**' -o target/coverage/html

grcov . --binary-path ./target/debug/ -s . -t html --branch --ignore-not-existing --keep-only '**/crates/**' --excl-start "#\[cfg\(test\)\]" --ignore '**/crates/testenv/**' --ignore '**/tests/**' --ignore '**/examples/**' -o target/coverage/html

cargo +nightly -q llvm-cov --doctests --branch --all --ignore-filename-regex "(example-crates/*|crates/testenv/*)" --all-features --open

¹

Possible solutions

• Include the following flags to the grcov command in the ci coverage job:

  • Add --ignore "**/crates/testenv/** flag: exclude testenv crate from coverage score.
  • Add --excl-start "#\[cfg\(test\)\]" flag: as long as BDK places unit test in module at the bottom of the files, this will exclude all test code from coverage score.
  • Add --excl-br-line "^\s*((debug_)?assert(_eq|_ne)?!|#\[derive\()" flag: ignore boilerplate code that can be confused as sources of branches.
  • Add --excl-line "^\s*\S+\s*(=>)?\s*unreachable!.*" flag: ignore unreachable code (not more than 3 lines in this repository codebase).

• Replace grcov by cargo-llvm-cov

Use case
Improve signal of test coverage score.

Additional context

I was experimenting with coverage tools in the ci last week, and discovered some issues with the local reporting of grcov. At the end I decided with cargo-llvm-cov for bdk-sp because it was the easiest way to avoid some false positives that I discovered with grcov, like misreporting lines under docstrings as not tested or marking test code as part of the overall coverage.

During today's call I mentioned the concern as bdk main repo is using grcov.
We discovered that crates/testenv wasn't being excluded from coveralls reports, and by later exploring them, I discovered other false positives, like test being counted towards the final coverage score.

I also tested against cargo-llvm-cov using nightly toolchain and the score was even lower.