Test failures in 12.1.1

[jsoo1]

Hello!

I am running into test failures like the following when updating to 12.1.1 from 11.x.

failures:

---- binary::after_match1_explicit stdout ----
thread 'binary::after_match1_explicit' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1:The Project Gutenberg EBook of A Study In Scarlet, by Arthur Conan Doyle
Binary file matches (found "\u{0}" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1:The Project Gutenberg EBook of A Study In Scarlet, by Arthur Conan Doyle
Binary file matches (found "\0" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/binary.rs:58:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- binary::after_match1_stdin stdout ----
thread 'binary::after_match1_stdin' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1:The Project Gutenberg EBook of A Study In Scarlet, by Arthur Conan Doyle
Binary file matches (found "\u{0}" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1:The Project Gutenberg EBook of A Study In Scarlet, by Arthur Conan Doyle
Binary file matches (found "\0" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/binary.rs:69:5

---- binary::before_match1_explicit stdout ----
thread 'binary::before_match1_explicit' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file matches (found "\u{0}" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file matches (found "\0" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/binary.rs:245:5

---- binary::after_match1_implicit_binary stdout ----
thread 'binary::after_match1_implicit_binary' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
hay:1:The Project Gutenberg EBook of A Study In Scarlet, by Arthur Conan Doyle
Binary file hay matches (found "\u{0}" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
hay:1:The Project Gutenberg EBook of A Study In Scarlet, by Arthur Conan Doyle
Binary file hay matches (found "\0" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/binary.rs:90:5

---- binary::after_match1_implicit stdout ----
thread 'binary::after_match1_implicit' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
hay:1:The Project Gutenberg EBook of A Study In Scarlet, by Arthur Conan Doyle
WARNING: stopped searching binary file hay after match (found "\u{0}" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
hay:1:The Project Gutenberg EBook of A Study In Scarlet, by Arthur Conan Doyle
WARNING: stopped searching binary file hay after match (found "\0" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/binary.rs:45:5

---- binary::before_match1_implicit_binary stdout ----
thread 'binary::before_match1_implicit_binary' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file hay matches (found "\u{0}" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file hay matches (found "\0" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/binary.rs:258:5

---- binary::after_match2_implicit stdout ----
thread 'binary::after_match2_implicit' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
hay:1:The Project Gutenberg EBook of A Study In Scarlet, by Arthur Conan Doyle
WARNING: stopped searching binary file hay after match (found "\u{0}" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
hay:1:The Project Gutenberg EBook of A Study In Scarlet, by Arthur Conan Doyle
WARNING: stopped searching binary file hay after match (found "\0" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/binary.rs:205:5

---- binary::before_match2_explicit stdout ----
thread 'binary::before_match2_explicit' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file matches (found "\u{0}" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file matches (found "\0" byte around offset 9741)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/binary.rs:293:5

---- misc::binary_convert_mmap stdout ----
thread 'misc::binary_convert_mmap' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file matches (found "\u{0}" byte around offset 3)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file matches (found "\0" byte around offset 3)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/misc.rs:1016:5

---- misc::binary_convert stdout ----
thread 'misc::binary_convert' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file matches (found "\u{0}" byte around offset 3)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file matches (found "\0" byte around offset 3)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/misc.rs:1006:5

---- misc::unrestricted3 stdout ----
thread 'misc::unrestricted3' panicked at '
printed outputs differ!

expected:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file hay matches (found "\u{0}" byte around offset 3)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

got:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Binary file hay matches (found "\0" byte around offset 3)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
', tests/misc.rs:793:5


failures:
    binary::after_match1_explicit
    binary::after_match1_implicit
    binary::after_match1_implicit_binary
    binary::after_match1_stdin
    binary::after_match2_implicit
    binary::before_match1_explicit
    binary::before_match1_implicit_binary
    binary::before_match2_explicit
    misc::binary_convert
    misc::binary_convert_mmap
    misc::unrestricted3

test result: FAILED. 244 passed; 11 failed; 0 ignored; 0 measured; 0 filtered out

This is for packaging in gnu guix, and the package definition looks like the following:

(define-public ripgrep
  (package
    (name "ripgrep")
    (version "12.1.1")
    (source
     (origin
       (method url-fetch)
       (uri (crate-uri "ripgrep" version))
       (file-name
        (string-append name "-" version ".tar.gz"))
       (sha256
        (base32
         "1grfi0j9zczzipipc21lkdbqmd2lvy2wlqy65fy4sckqvix5amdr"))))
    (build-system cargo-build-system)
    (arguments
     `(#:cargo-inputs
       (("rust-bstr" ,rust-bstr-0.2)
        ("rust-clap" ,rust-clap-2)
        ("rust-grep" ,rust-grep-0.2)
        ("rust-ignore" ,rust-ignore-0.4)
        ("rust-jemallocator" ,rust-jemallocator-0.3)
        ("rust-lazy-static" ,rust-lazy-static-1)
        ("rust-log" ,rust-log-0.4)
        ("rust-num-cpus" ,rust-num-cpus-1)
        ("rust-regex" ,rust-regex-1)
        ("rust-serde-json" ,rust-serde-json-1)
        ("rust-termcolor" ,rust-termcolor-1))
       #:cargo-development-inputs
       (("rust-serde" ,rust-serde-1)
        ("rust-serde-derive" ,rust-serde-derive-1))
       #:modules ((ice-9 match)
                  (guix build cargo-build-system)
                  (guix build utils))
       #:phases
       (modify-phases %standard-phases
         (add-after 'build 'install-manpage
           ;; NOTE: This is done before 'check so that there's only one output
           ;; directory with the man page.
           (lambda* (#:key outputs #:allow-other-keys)
             (match (find-files "target" "^rg\\.1$")
               ((manpage)
                (install-file manpage (string-append
                                        (assoc-ref outputs "out")
                                        "/share/man/man1"))))
             #t)))
       #:features '("pcre2")))
    (native-inputs
     `(("asciidoc" ,asciidoc)
       ("pcre2" ,pcre2)
       ("pkg-config" ,pkg-config)))
    (home-page "https://github.com/BurntSushi/ripgrep")
    (synopsis "Line-oriented search tool")
    (description
     "ripgrep is a line-oriented search tool that recursively searches
your current directory for a regex pattern while respecting your
gitignore rules.")
    (license (list license:unlicense license:expat))))

[BurntSushi]

This isn't failing on master or the 12.1.1 tag right? The guix package stuff is greek to me. Please provide the exact commands you're using (starting from how you get the source) that reproduces the failures.

My guess is that your Cargo.lock is mismatched from the 12.1.1 release.

[jsoo1]

The guix package stuff is greek to me.

I figured as much. Apologies for the noise.

Please provide the exact commands you're using (starting from how you get the source) that reproduces the failures.

My guess is that your Cargo.lock is mismatched from the 12.1.1 release.

The Cargo.lock is definitely the primary suspect since we re-vendor most crates from the system defined ones. What crate should I be looking at most likely? Here are the build log and the environment variables for the build.

build.log

build-env-vars.log

The test command is cargo test --release the build command is cargo build --release.

Most likely it is a crate mismatch I think.

[jsoo1]

Ah, I think we also use --features pcre2

[BurntSushi]

Hmmm that build log doesn't really help me. I'm after a reproduction here. A reproduction should be a sequence of steps that I can follow, using git and cargo commands, to reproduce your test failures.

My suspicion is that the bstr crate version mismatch is the problem. ripgrep 12.1.1 shipped with bstr 0.2.13. But your build log suggests it is trying to use 0.2.15.

In general, this is not a supported configuration. I can't really support ripgrep being built with arbitrary versions of crates. In this particular case, since I'm the author of bstr too, I was able to easily identify the problem. But it may not be so easy in other cases. In particular, bstr changed its debug output of a byte string, which ripgrep also uses. Writing unit tests against such behavior isn't the best idea---since debug output is not covered under semver---but it's not typically a problem in practice.

Note that when filing future bug reports against any Rust project, it would be good to lead with the fact that you aren't using the project's Cargo.lock. This is really critical. I only jumped to that initially because I've seen these kinds of bug reports from distro maintainers before. They have burned a lot of time (for all involved) in the past. This is why I'm now more insistent on getting a repro outside the context of their package manager. Short of that, noting that Cargo.lock is being ignored is really really really critical difference between your build environment and the typical build environment of a Rust programmer.

[jsoo1]

Totally, totally understand. I don't personally endorse the manner in which the Cargo.lock is disregarded, apologies.

Note that when filing future bug reports against any Rust project, it would be good to lead with the fact that you aren't using the project's Cargo.lock. This is really critical. I only jumped to that initially because I've seen these kinds of bug reports from distro maintainers before. They have burned a lot of time (for all involved) in the past. This is why I'm now more insistent on getting a repro outside the context of their package manager. Short of that, noting that Cargo.lock is being ignored is really really really critical difference between your build environment and the typical build environment of a Rust programmer.

Understood! I apologize again, I don't mean to waste your time. I was looking more for a place to ask a question which github seemed to suggest with the Q & A option in the issues menu.

Here's the reproduction:

git clone ripgrep
git checkout 12.1.1
rm Cargo.lock
cargo test --release --features pcre2

Changing the version bounds of bstr in Cargo.toml to bstr = "=0.2.12", and running rm Cargo.lock && cargo test --release --features pcre2 indeed fixes the issue. Thank you so much for your time and dedication. You are one of my rustacean heroes.

[BurntSushi]

Yeah that sounds about right to me. Thanks for your kind words. :-)

[jsoo1]

Thank you for being so fast! Have a nice day!