Report on nocov comments usage

[maelle]

This would make editors and reviewers aware of what isn't tested.

[mpadge]

It could, however, be quite verbose, as there can be valid reasons for using those (like handling API errors which can't be mocked). There are also lots of options, including .covrignore files, manual edits to test-coverage workflow files to add line_exclusions, single-line # nocov, and multi-line start and end delimiters. I suspect those two considerations mean that this would require:

1. An additional separate section to the pkgcheck report. In that case, it should only be activated by some kind of threshold behaviour, requiring an additional subjective decision; and
2. Potential implementation of ways to determine appropriate coverage ignores. Most importantly, we would need to automatically ignore coverage ignore of any bundled source code. That is very common, and usually but not always includes distinct headers which it may be possible to parse and auto-identify. But any general implementation would be very tricky.

I guess what I'm saying is that, although this might seem like a good and useful idea, implementing is going to be very tricky, and the result will be very likely to generate too much noise.

[maelle]

Could it be a separate report linked to from the general report?

Or maybe exploring exclusions should be a separate thing listed in editor/reviewer checklists.

https://github.com/r-lib/covr/blob/d8d33b41b6b013a3e96ea04cb6d1bf434c4822d2/R/exclusions.R#L52C1-L52C8

[mpadge]

Or we could just add an extra 👀 check result if total exclusions when running tests exceed some threshold, and just state the % excluded lines? (And presuming that covr.exclude_pattern/start/stop options remain at default values.)

[maelle]

And presuming that covr.exclude_pattern/start/stop options remain at default values.

Wouldn't using the same code as covr itself prevents problems in that case anyway?