GitError when running benchmarkpkg with GitLab CI

[danielmatz]

I'm trying to use PkgBenchmark with one of the private packages we've developed at my company. We have a private registry. The registry and packages are stored on a self-hosted GitLab instance.

I have a script that is more or less doing the following:

using PkgBenchmark

pkg_root = abspath(@__DIR__, "..") # this script lives in the benchmark subdirectory, so I'm pointing up one level

function benchmark_config(; kw...)
    BenchmarkConfig(juliacmd=`julia -O3`,
                    env=Dict("JULIA_NUM_THREADS" => 1);
                    kw...)
end

target = benchmarkpkg(pkg_root, benchmark_config())
baseline = benchmarkpkg(pkg_root, benchmark_config(id="8d33882")) # this is line 26 in the stack trace below

I have a hard-coded id there just to test things out. Eventually, I'll use git merge-base to figure out the correct id.

I can run this locally on my Mac just fine. When I try to call this using GitLab CI, with a runner on a Linux machine, I get the error below.

818 ERROR: LoadError: GitError(Code:ERROR, Class:Index, invalid path: '.julia/registries/EG/')
819 Stacktrace:
820   [1] macro expansion
821     @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/LibGit2/src/error.jl:110 [inlined]
822   [2] add!(idx::LibGit2.GitIndex, files::String; flags::UInt32)
823     @ LibGit2 /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/LibGit2/src/index.jl:107
824   [3] add!
825     @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/LibGit2/src/index.jl:106 [inlined]
826   [4] (::LibGit2.var"#157#160"{GitRepo})(idx::LibGit2.GitIndex)
827     @ LibGit2 /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/LibGit2/src/LibGit2.jl:908
828   [5] with(f::LibGit2.var"#157#160"{GitRepo}, obj::LibGit2.GitIndex)
829     @ LibGit2 /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/LibGit2/src/types.jl:1150
830   [6] with
831     @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/LibGit2/src/types.jl:1156 [inlined]
832   [7] snapshot(repo::GitRepo)
833     @ LibGit2 /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/LibGit2/src/LibGit2.jl:906
834   [8] transact(f::PkgBenchmark.var"#39#40"{PkgBenchmark.var"#do_benchmark#22"{Nothing, Bool, Bool, Nothing, String, String, BenchmarkConfig, Bool, String}, String, String}, repo::GitRepo)
835     @ LibGit2 /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/LibGit2/src/LibGit2.jl:953
836   [9] _withcommit(f::PkgBenchmark.var"#do_benchmark#22"{Nothing, Bool, Bool, Nothing, String, String, BenchmarkConfig, Bool, String}, repo::GitRepo, commit::String)
837     @ PkgBenchmark /scratch/00/gitlab-runner/Z_-sry_E/0/Genesis/Genesis.jl/.julia/packages/PkgBenchmark/N88Tj/src/util.jl:17
838  [10] benchmarkpkg(pkg::String, target::BenchmarkConfig; script::Nothing, postprocess::Nothing, resultfile::Nothing, retune::Bool, verbose::Bool, logger_factory::Nothing, progressoptions::Nothing, custom_loadpath::String)
839     @ PkgBenchmark /scratch/00/gitlab-runner/Z_-sry_E/0/Genesis/Genesis.jl/.julia/packages/PkgBenchmark/N88Tj/src/runbenchmark.jl:139
840  [11] benchmarkpkg(pkg::String, target::BenchmarkConfig)
841     @ PkgBenchmark /scratch/00/gitlab-runner/Z_-sry_E/0/Genesis/Genesis.jl/.julia/packages/PkgBenchmark/N88Tj/src/runbenchmark.jl:56
842  [12] top-level scope
843     @ /.scratch00/gitlab-runner/Z_-sry_E/0/Genesis/Genesis.jl/benchmark/run_ci.jl:26
844 in expression starting at /.scratch00/gitlab-runner/Z_-sry_E/0/Genesis/Genesis.jl/benchmark/run_ci.jl:26

That "EG" registry is our private registry. Line 26 of run_ci.jl is the second call to benchmarkpkg above. It's like it is trying to operate on the registry's repo instead of my package's repo.

I'm using Julia 1.6.1 on both systems, and I'm using v0.2.11 of PkgBenchmark.

I set JULIA_DEPOT_PATH to force the runner to start with a fresh depot. I've looked at the environment variables that GitLab CI is setting, but nothing jumped out at me.

I can try to reduce this down to a simpler demonstration of the problem, but I thought I'd post what I have so far, in case something jumps out to someone.

[danielmatz]

I was able to manually run my script on the Linux machine. So, the issue does seem to be confined to GitLab CI...

[danielmatz]

I was able to work around the issue by using LibGit2 to check out the appropriate version of the code myself.

using PkgBenchmark
import LibGit2

pkg_root = abspath(@__DIR__, "..") # this script lives in the benchmark subdirectory, so I'm pointing up one level

function benchmark_config(; kw...)
    BenchmarkConfig(juliacmd=`julia -O3`,
                    env=Dict("JULIA_NUM_THREADS" => 1);
                    kw...)
end

target = benchmarkpkg(pkg_root, benchmark_config())
repo = LibGit2.GitRepo(pkg_root)
LibGit2.checkout!(repo, baseline_hash)
baseline = benchmarkpkg(pkg_root, benchmark_config())
LibGit2.checkout!(repo, target_hash)

This makes me think even more that PkgBenchmark is passing the wrong repo to a LibGit2 call somewhere...