performance regression in 1.12.0-beta2 when summing over iterator

[matthias314]

I'm observing a 2.5x slowdown when summing over an iterator. The code

using Chairmarks

struct A end
struct B end
struct C end
struct D end
struct E end
struct F end

const U = Union{A, B, C, D, E, F}

f(a::A) = 1
f(b::B) = 2
f(c::C) = 3
f(d::D) = 4
f(e::E) = 5
f(f::F) = 6

v = U[rand((A(), B(), C(), D(), E(), F())) for _ in 1:1000];

gives

julia> @b sum(f, $v), sum(f, x for x in $v)
(431.909 ns, 1.771 μs)  # 1.11.5
(441.500 ns, 4.083 μs)  # 1.12.0-beta2

Julia Version 1.12.0-beta2
Commit dd74040f22b (2025-04-25 12:03 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 4 × Intel(R) Core(TM) i3-10110U CPU @ 2.10GHz
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, skylake)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 4 virtual cores)

[jakobnissen]

I cannot reproduce - sorry, I CAN reproduce. It produces different machine code with the 1.11 version having one more branch per iteration. There is nothing obviously wrong with the 1.12 machine code - it inlines, is fully inferred, and union-split and there are no extra error branches or such.

[giordano]

julia> @b sum(f, $v), sum(f, x for x in $v)
(686.000 ns, 352.679 ns) # 1.11.5
(679.500 ns, 434.269 ns) # 1.12.0-beta2
(657.051 ns, 79.051 ns) # 1.13.0-DEV.468

Julia Version 1.13.0-DEV.468
Commit 2270fcdcc65 (2025-04-29 15:50 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 8 × Apple M1
  WORD_SIZE: 64
  LLVM: libLLVM-19.1.7 (ORCJIT, apple-m1)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 4 virtual cores)

This is a lot faster on master apparently?

[sgaure]

I've bisected the speedup on master to #52850.

[dpanigo]

I would like to add that I also experience a runtime regression starting from Julia 1.9.4 — which seems to get worse in Julia 1.10 LTS (partially reverted in 1.11 release), and worsens again in the latest beta (1.12).

To test this, I ran the following PowerShell script to compare execution times across Julia versions:

# Ruta al script de Julia
$JULIA_SCRIPT = 'C:\gsreg.jl'

# Lista de versiones
$versions = @(
    '+1.8.5',
    '+1.9.4',
    '+lts',
    '+release',
    '+beta'
)

# Archivo de resultados
$resultsFile = 'timing_results.txt'
"Julia version timings for $JULIA_SCRIPT" | Out-File -FilePath $resultsFile
"---------------------------------------" | Out-File -FilePath $resultsFile -Append

# Loop sobre versiones
foreach ($version in $versions) {
    Write-Host "Running Julia $version..."
    $time = Measure-Command {
        & julia $version -p auto $JULIA_SCRIPT
    }
    $realTime = '{0:mm\:ss\.fff}' -f $time
    "Julia $version - Time: $realTime" | Out-File -FilePath $resultsFile -Append
}

"---------------------------------------" | Out-File -FilePath $resultsFile -Append
Write-Host "Timing results saved to $resultsFile"

This script calls the following Julia code:

# This is the gsreg.jl file
using GlobalSearchRegression, DataFrames, Distributions
# Insert here your working directory path
data = DataFrame(rand(100, 21), :auto)
headers = [ :y ; [ Symbol("x$i") for i = 1:size(data,2) - 1 ] ]
rename!(data, headers)
gsreg("y x*", data; parallel=16)
exit()

I obtained these results:

Julia version timings for C:\gsreg.jl
---------------------------------------
Julia +1.8.5 - Time:      01:04.563
Julia +1.9.4 - Time:      01:03.225
Julia +lts - Time:        01:23.469
Julia +release - Time:    01:12.619
Julia +beta - Time:       01:26.860
---------------------------------------

Observations:

In my case, the slowdown seems to start with 1.10 LTS, is partially improved in 1.11, but worsens again in 1.12 beta 4. The observed regressions are not massive, but clearly measurable and consistent across multiple runs. The task performs one million OLS estimations with artificial data, so it might be related to the underlying discussion here. I hope this helps contribute to the diagnosis.

==================================================================================
Timing update using 1.13-DEV (nightly) version:
Julia 1.13 partially reverts the runtime regression observed in 1.10 and 1.12 (Julia +nightly - Time: 01:13.197) but it is still 10% slower than 1.8.5 and 1.9.4 versions.