-Zrustdoc-scrape-examples=all gives an error that some examples are missing

[jyn514]

Problem

I think the issue is that cargo isn't waiting for some rustdoc processes to complete?

$ rustup run nightly $CARGO_TARGET_DIR/debug/cargo doc -Zrustdoc-scrape-examples=all --offline
   Compiling cargo-deadlinks v0.8.1 (/home/joshua/src/rust/cargo-deadlinks)
 Documenting cargo-deadlinks v0.8.1 (/home/joshua/src/rust/cargo-deadlinks)
error: failed to load examples: No such file or directory (os error 2) (for path /home/joshua/.local/lib/cargo/target/debug/deps/cargo-deadlinks-8d06e4def16d1602.examples)

error: could not document `cargo-deadlinks`

Steps

1. git clone https://github.com/deadlinks/cargo-deadlinks/ && cd cargo-deadlinks
2. cargo doc -Zrustdoc-scrape-examples=all

Possible Solution(s)

No response

Notes

No response

Version

cargo 1.58.0 (built from source on b4ab730)

[alexcrichton]

cc @willcrichton

[willcrichton]

@jyn514 does this issue happen consistently for you? I couldn't reproduce it after ~5 build attempts.

If you can repro it, can you run with -v and paste the logs?

[jyn514]

Hmm, it reproduces on one laptop but not the other ... I wonder if it's related to my shared CARGO_TARGET_DIR? I'll get you the logs this afternoon.

[Emilgardis]

I get this consistently when running with -Zrustdoc-scrape-examples=all and using another target-dir specified as --target-dir target/extra, solution is to run twice.

The first build seems to fail when compiling the first -p krate and before compiling all workspace members, the second build completes by first building the workspace members and then compiling and documenting the -p krate

edit: A few more tries now with cleans inbetween, can't reproduce it fully anymore, and a second run doesn't always solve it, not sure what makes it not work. Have replicated this in CI, see https://github.com/twitch-rs/twitch_api/actions/runs/3306813105/jobs/5457978565 (gist)

[willcrichton]

Thanks for the report @Emilgardis. I tried reproducing by cloning the twitch_api repo and running cargo xtask doc, but I encountered no errors. I'll see if I can reproduce it another way.

[Emilgardis]

Ok!

If I could be of assistance in the reproduction of the bug or with producing logs, I opened https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/debugging.20.22.60--scrape-examples.3Dall.60.20failed.20to.20load.20examples.22 earlier

[willcrichton]

Actually I think I see the issue. Your logs show that channel_information-{some hash}.examples was generated, but then Cargo tried to read channel_information-{a different hash}.examples later. So there's some kind of issue in selecting which hash to write/read from. I'll keep investigating this.