cargo metadata --offline requires Internet access

[0xngold]

Problem

Actual behavior: running cargo metadata --offline in a dependency of the primary crate tries to access the Internet. If the host has no access, this fails with a network error.
Expected behavior: cargo metadata --offline does not attempt to access the Internet & returns a limited set of fields that can be calculated offline.

Weird complicating factor: if the crate in question is vendored as a path dependency, cargo metadata works without the --offline switch.

Error example

error: proc macro panicked
 --> C:\Users\some_user\.cargo\registry\src\-b20c50870c528e96\windows-0.3.1\build.rs:2:5
2 |     windows_macros::build!();
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^

  = help: message: Cargo metadata \\?\C:\src\.rustup\toolchains\stable-x86_64-pc-windows-msvc\bin\cargo.exe did not contain `workspace_root` key. Got stdout . Got stderr     
          Updating crates.io index
          warning: spurious network error (2 tries remaining): failed to send request: The server name or address could not be resolved
          ; class=Os (2)
          warning: spurious network error (1 tries remaining): failed to send request: The server name or address could not be resolved
          ; class=Os (2)
          error: failed to get `const-sha1` as a dependency of package `windows v0.3.1 (C:\Users\some_user\.cargo\registry\src\-b20c50870c528e96\windows-0.3.1)`

          Caused by:
            failed to load source for dependency `const-sha1`

          Caused by:
            Unable to update registry `https://github.com/rust-lang/crates.io-index`

          Caused by:
            failed to fetch `https://github.com/rust-lang/crates.io-index`

          Caused by:
            failed to send request: The server name or address could not be resolved
            ; class=Os (2)

          .
   Ran from "C:\\Users\\some_user\\.cargo\\registry\\src\\-b20c50870c528e96\\windows-0.3.1"

error: aborting due to previous error

Steps

1. Create a binary crate C that depends on windows-rs.
2. Use cargo local-registry to create a local registry for C's dependencies.
3. Try to build C with the local registry replacing crates.io, and no Internet access. The build should fail with a cryptic error from the windows-rs build. To reproduce the error message I have above, modifying this line inside the local registry's copy of windows_winmd crate. There is almost certainly a simpler repro, but I suspect an exact repro won't be strictly necessary to address this bug.

Possible Solution(s)

When we make windows-rs a path dependency, it allows cargo metadata to run in that crate without errors while offline. Whatever this code path is, can do the same thing in the local registry case? My suspicion is that C's lockfile somehow applies to the cargo metadata command when windows-rs is a path dependency.

Notes

Output of cargo version:

> cargo version
cargo 1.48.0 (65cbdd2dc 2020-10-14)

[Revantus]

Hi @0xngold, it looks like this error is originating from windows-rs here. When the build script is triggered for windows-rs, it makes another cargo call separate of the one you invoked and it does not respect the --offline flag.

[0xngold]

Hi @Revantus, while the windows-rs crate at ToT doesn't currently use the --offline flag, I modified it to do so in a local registry for testing. Even with the --offline flag, the error still occurs. As far as I can tell, there is just one instance of the function (it seems the crate authors did some refactoring recently).

[Revantus]

With the most recent update to windows-rs I'm unable to replicate the problem. I didn't dig too deep, but their refactoring and a recent workaround might have fixed it.

Can you try these steps? These are what I used to try to replicate the issue on WLS and Windows native.

$ mkdir test_offline
$ cd test_offline
$ mkdir c_local_registry
$ cargo new c
$ cd c
-- add 'windows = "0.5.0"' to the dependencies and build-dependencies in the Cargo.toml --
$ cargo generate-lockfile
$ cd ../c_local_registry
$ cargo local-registry -s ../c/Cargo.lock
$ cd ../c
$ mkdir .cargo
-- add the output from cargo local-registry into .cargo/config --
----------------------------SAMPLE---------------------------------
    [source.crates-io]
    registry = 'https://github.com/rust-lang/crates.io-index'
    replace-with = 'local-registry'

    [source.local-registry]
    local-registry = '/home/some_user/test_offline/c_local_registry/'    # WSL
    local-registry = 'C:\Users\some_user\test_offline\c_local_registry\'  # Windows Native
--------------------------SAMPLE_END-------------------------------

$ cargo build --offline

------OUTPUT------ WSL
$ cargo build --offline
   Unpacking const-sha1 v0.2.0 (registry `/home/some_user/test_offline/c_local_registry`)
   Unpacking proc-macro2 v1.0.24 (registry `/home/some_user/test_offline/c_local_registry`)
   Unpacking quote v1.0.9 (registry `/home/some_user/test_offline/c_local_registry`)
   Unpacking squote v0.1.2 (registry `/home/some_user/test_offline/c_local_registry`)
   Unpacking syn v1.0.64 (registry `/home/some_user/test_offline/c_local_registry`)
   Unpacking unicode-xid v0.2.1 (registry `/home/some_user/test_offline/c_local_registry`)
   Unpacking windows v0.5.0 (registry `/home/some_user/test_offline/c_local_registry`)
   Unpacking windows_gen v0.5.0 (registry `/home/some_user/test_offline/c_local_registry`)
   Unpacking windows_gen_macros v0.5.0 (registry `/home/some_user/test_offline/c_local_registry`)
   Unpacking windows_macros v0.5.0 (registry `/home/some_user/test_offline/c_local_registry`)
   Compiling .....
   Compiling c v0.1.0 (/home/some_user/test_offline/c
    Finished dev [unoptimized + debuginfo] target(s) in 17.76s
----OUTPUT_END----

------OUTPUT------ Powershell (Windows Native)
$ cargo build --offline
   Unpacking const-sha1 v0.2.0 (registry `C:\Users\some_user\test_offline\c_local_registry`)
   Unpacking proc-macro2 v1.0.24 (registry `C:\Users\some_user\test_offline\c_local_registry`)
   Unpacking quote v1.0.9 (registry `C:\Users\some_user\test_offline\c_local_registry`)
   Unpacking squote v0.1.2 (registry `C:\Users\some_user\test_offline\c_local_registry`)
   Unpacking syn v1.0.64 (registry `C:\Users\some_user\test_offline\c_local_registry`)
   Unpacking unicode-xid v0.2.1 (registry `C:\Users\some_user\test_offline\c_local_registry`)
   Unpacking windows v0.5.0 (registry `C:\Users\some_user\test_offline\c_local_registry`)
   Unpacking windows_gen v0.5.0 (registry `C:\Users\some_user\test_offline\c_local_registry`)
   Unpacking windows_gen_macros v0.5.0 (registry `C:\Users\some_user\test_offline\c_local_registry`)
   Unpacking windows_macros v0.5.0 (registry `C:\Users\some_user\test_offline\c_local_registry`)
   Compiling .....
   Compiling c v0.1.0 (C:\Users\some_user\test_offline\c)
    Finished dev [unoptimized + debuginfo] target(s) in 17.76s
----OUTPUT_END----

From what I'm seeing, Cargo is only unpacking the already downloaded and cached crates and not accessing the internet.
I think this issue can be closed.

[0xngold]

It looks like the downstream worked around the problem with this change. That change lagged the published version I tested with to file the bug report. I retested with 0.5.0 and it is working offline 😃.

Is this workaround the desired long term state for cargo? Note that they're using --no-deps to get --offline behavior where --offline didn't work.

[ehuss]

where --offline didn't work.

cargo metadata --offline should work, supposing either of the following two are done:

• Use --filter-platform to only show metadata for a specific platform that has been downloaded.
• Do a complete fetch with cargo fetch to fetch all platform dependencies.

cargo metadata without any flags needs all platform-specific dependencies to be downloaded, which I suppose is what you ran into?

[0xngold]

Re cargo metadata --offline only working in particular situations, that sounds like what we ran into.

cargo fetch is not necessarily compatible with the idea of a local registry. I think we'd have to switch off (or supplement) local registries to use it. From what I've read, there are some elements which don't seem to be part of local registries, but are available from crates.io, and that's what the metadata command needs. This seems potentially incorrect to me as some of the items I saw it looking for are inside the crate package, but I digress.

I think this bug might be best scoped down to more of a UX issue [0]. In short, the real bug here is: when cargo metadata --offline "fails" and needs Internet access, there is currently no meaningful error. The failure modes also, at least as far as I could find, weren't documented.

[0]: why scope down? I think there's a level of enhancement that is being requested here as well (local registries should enable all offline operations), but that is probably a feature request.