`cargo test --all-targets` does not run doc tests

[thomaseizinger]

As per title cargo test --all-targets does not run the doc tests on a crate. This is very surprising behaviour. I found this and it looks like running doc tests is done by rustdoc which is apparently not invoked when doing --all-targets.

[dwijnand]

An attempt to fix this was made in #6671, but there's a bit of a mismatch between cargo test and doc tests that makes the issue complicated, so the PR didn't land.

[thomaseizinger]

Maybe it is worth to add some context to this issue.

We ran into this because we use cargo test --all-targets on Travis and usually cargo test locally.
A not-compiling doc test sneaked into a last-minute update of a PR but the build was fine since we use --all-targets on Travis.
Later on, someone complained that we have a broken master because of a non-compiling doc test.

I read some of the discussion in the #6671 and TBH, the whole --doc is a mode would be SUPER confusing.
As a user, I expect cargo test to test LESS than cargo test --all-targets since the latter has all in the name.
In addition, the documentation (--help) lists doc tests next to all the other targets.

[glts]

This means the command to run absolutely all tests everywhere is cargo test --workspace, not cargo test --workspace --all-targets, right?

[ehuss]

This means the command to run absolutely all tests everywhere is cargo test --workspace, not cargo test --workspace --all-targets, right?

It depends on the structure of your packages, but in general there is currently no single command to "test everything". cargo test is intended to be the usual command. If you also want to test benchmarks and examples, you can either mark them with test=true in Cargo.toml, or run two commands (cargo test --all-targets and cargo test --doc).