network zlib stream error confusion

[ehuss]

Problem
On rust-lang/rust CI, I've been noticing this error happening a lot lately:

    Updating crates.io index
error: failed to get `cc` as a dependency of package `bootstrap v0.0.0 (D:\a\1\s\src\bootstrap)`

Caused by:
  failed to fetch `https://github.com/rust-lang/crates.io-index`

Caused by:
  error inflating zlib stream; class=Zlib (5)

I think the CI has been having network problems in general, but this error in particular seems a little confusing to me. I don't know what would cause this error. I think it might be helpful to understand this error case a little better.

Here is the corresponding error in libgit2.

Should this error class be added to the spurious error list? What can possibly cause this to happen?

Notes

cargo 1.46.0-beta (4f74d9b2a 2020-07-08)

[alexcrichton]

This looks reminiscent of #8258 which was fixed in libgit2/libgit2#5536. Rust 1.45 did not include that PR, but 1.46 (beta) and nightly both have the fix. Do you know if these errors are showing up using Cargo from 1.45? (bootstrap not updated?)

[ehuss]

I'm pretty sure it is the version posted above cargo 1.46.0-beta (4f74d9b2a 2020-07-08)

extracting D:\a\1\s\build\cache\2020-07-16\cargo-beta-i686-pc-windows-msvc.tar.xz

https://dev.azure.com/rust-lang/rust/_build/results?buildId=35201&view=logs&j=339a807e-05eb-50a3-f69a-0d7a3a5b2cef&t=1f87e16f-7721-57ff-dca8-75b52e5f4eac

[alexcrichton]

Bah.

It probably wouldn't hurt to throw this onto the list of spurious errors, but I agree it'd be good to understand. AFAIK zlib inflate errors indicate a corrupt data (e.g. not zlib) getting fed to the zlib inflater. That previous issue was due to a bug in libgit2 (valid zlib stream, invalidly fed), and this may also be that. This'll be really hard to track down though without a reliable reproduction.

Skimming libgit2's commit history I don't see much related to this since that original fix, so I don't think we can get lucky by "simply update the git repo".

[ehuss]

Another instance: https://dev.azure.com/rust-lang/rust/_build/results?buildId=35201&view=logs&j=339a807e-05eb-50a3-f69a-0d7a3a5b2cef&t=1f87e16f-7721-57ff-dca8-75b52e5f4eac

I wonder if there is any significance that these cases happen on Windows i686 (i686-mingw-1 and i686-msvc-1).

[alexcrichton]

Hm the fact that it was retried there and it didn't help anything makes me think that it's not a spurious failure...

[ehuss]

Hm, looking at other instances:
rust-lang/rust#73210 (comment)
https://dev.azure.com/rust-lang/e71b0ddf-dd27-435a-873c-e30f86eea377/_build/results?buildId=35451
Failed in three builders: i686-gnu-opt i686-msvc-1 dist-i686-msvc

rust-lang/rust#74245 (comment)
i686-gnu

rust-lang/rust#74245 (comment)
i686-gnu

rust-lang/rust#74245 (comment)
i686-gnu-opt
i686-msvc-1
dist-i686-mingw

rust-lang/rust#74611 (comment)
i686-mingw-2

rust-lang/rust#74495 (comment)
i686-mingw-2

rust-lang/rust#74520 (comment)
i686-msvc-1

rust-lang/rust#74219 (comment)
i686-mingw-1

Raising priority since this seems important.

[mati865]

It cannot be spurious, it affects probably all i686 platforms.