Backports for 1.12.0-rc1 #58655
Merged
Backports for 1.12.0-rc1 #58655
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This code was checking for the old edge type (`MethodInstance`) instead of the new one (`CodeInstance`), causing duplicate non-invoke edges to accumulate in our `backedges`. (cherry picked from commit cf875c1)
Since the caller CodeInstance is always part of the identity that we are de-duplicating on, this makes the linear scan much faster than it is in `gf.c` (cherry picked from commit 366a224)
…Table` These are restored in their entirety by staticdata.jl, so there's no need to serialize them. Dropping them has the additional advantage of making it unnecessary to de-duplicate edges in `gf.c` (cherry picked from commit 5c66152)
On my system, this saves ~500 ms when loading CairoMakie (and all dependent packages) (cherry picked from commit bf725f1)
(cherry picked from commit 7525568)
DRY code somewhat by consolidating builtin declarations to use a table or macro definition to auto-generate the required reflection metadata. Mostly NFC, but does include a couple bugfixes caused by the consistency this enforces. Adding a new `Builtin` is now simply a matter of declaring it in `builtin_proto.h` and defining it in `builtins.c` and any relevant declarations are handled automatically. (cherry picked from commit 907b201)
Instead of hiding the fragments of the method table in each TypeName, make one
variable `Core.GlobalMethods` with access to all methods. The need to split
them early for performance apparently had been long past since kwargs and
constructors were already using a single table and cache anyways. Some new
concepts introduced here:
- A single Method can now be added to multiple functions. So instead of using
eval in a for loop, we could define it just once (see example below).
- Several fields (`max_args`, `name`, and `backedges`) were moved from
MethodTable to their TypeName.
- TypeName currently has a (user-modifiable) field called `singletonname`. If
set to something other than `name`, it may be used for pretty printing of a
singleton object using its "canonical" (unmangled) name, particularly for
`function`.
- `Core.Builtin` method table entries are even more normal now, with valid
`sig` fields, and special logic to specifically prevent adding methods which
would become ambiguous with them (as that would violate the tfuncs we have
for them).
- `Core.GlobalMethods` is a `Base.Experimental.@MethodTable GlobalMethods`.
- Each `MethodTable` contains a separate `MethodCache` object for managing
fast dispatch lookups. We may want to use this for the `Method` field
containing the `invokes` list so that lookups there get more of the same
optimizations as global calls.
- Methods could be put into any number of different MethodTables (or none).
The `Method.primary_world` field is intended to reflect whether it is
currently put into the GlobalMethods table, and what world to use in the
GlobalMethods table for running its generator, and otherwise is meaningless.
- The lock for TypeName backedges is a single global lock now, in
`Core.GlobalMethods.mc`.
- The `backedges` in TypeName are stored on the "top-most" typename in the
hierarchy, to enable efficient lookup (although we might want to consider
replacing this entirely with a TypeMap). The "top-most" typename is the
typename of the type closest to Any, after union-splitting, which doesn't
have an intersection with Builtin (so Function and Any by implication
continue to not require scanning for missing backedges since it is not
permitted to add a Method applicable to all functions).
- Support for having backedges from experimental method tables was removed
since it was unsound and had been already replaced with staticdata.jl
several months ago.
- Documentation lookup for `IncludeInto` is fixed (previously attached only to
`Main.include` instead of all `include` functions).
Example: given this existing code in base/operators:
for op in (:+, :*, :&, :|, :xor, :min, :max, :kron)
@eval begin
($op)(a, b, c, xs...) = (@inline; afoldl($op, ($op)(($op)(a,b),c), xs...))
end
end
It could now instead be equivalently written as:
let ops = Union{typeof(+), typeof(*), typeof(&), typeof(|), typeof(xor), typeof(min), typeof(max), typeof(kron)}
(op::ops)(a, b, c, xs...) = (@inline; afoldl(op, (op)((op)(a,b),c), xs...))
end
Fixes #57560
(cherry picked from commit 1735d8f)
Introduce a new `jl_get_global_value` to do the new world-aware behavior, while preserving the old behavior for `jl_get_global`. Choose between `jl_get_global`, `jl_get_global_value`, and `jl_eval_global_var`, depending on what behavior is required. Also take this opportunity to fix some data race mistakes introduced by bindings (relaxed loads of jl_world_counter outside of assert) and lacking type asserts / unnecessary globals in precompile code. Fix #58097 Addresses post-review comment #57213 (comment), so this is already tested against by existing logic (cherry picked from commit 965d007)
The A15 was detected as M2; added codenames for easier future updates. Sources: - https://asahilinux.org/docs/hw/soc/soc-codenames/#socs - https://github.com/apple-oss-distributions/xnu/blob/main/osfmk/arm/cpuid.h - https://github.com/llvm/llvm-project/blob/main/llvm/lib/Target/AArch64/AArch64Processors.td#L428 Missing: - the M4 Pro and Max are missing (because they are missing from Apple's `cpuid.h`) Resolves #58278 --------- Co-authored-by: Christian Guinard <28689358+christiangnrd@users.noreply.github.com> (cherry picked from commit 7cb88d6)
(cherry picked from commit f5e983e)
(cherry picked from commit 347fb7c)
This makes two changes to the backdate-warning-turned-error (#58266): 1. Fix a bug where the error would only trigger the first time. We do only want to print once per process, but of course, we do want to error every time if enabled. 2. If we are in speculative execution context (generators and speculatively run functions during inference), always use the UndefVarError. Effects from these functions are not supposed to be observable, and it's very confusing if the printed warning goes away when depwarns are enabled. This is marginally more breaking, but the burden is on generated function authors (which already have to be world-age aware and are somewhat more regularly broken) and is consistent with other things that are stronger errors in pure context. Fixes #58648 (cherry picked from commit d2cc061)
This dodges the issue on my machine, let's see if it works for everyone. (cherry picked from commit dda37f9)
It turns out that there are two path types in applescript, and I had mixed two of them in my previous patch. Annoyingly, things seemed to work when editing locally, unsure why. (cherry picked from commit c759aa9)
Manage a single dictionary (keyed by TypeName) instead of scattering this info into each TypeName scattered across the system. This makes it much easier to scan the whole table when required and to split it up better, so that all kwcalls and all constructors don't end up stuck into just one table. While not enormous (or even the largest) just using the REPL and Pkg, they are clearly larger than intended for a linear scan: ``` julia> length(Type.body.name.backedges) 1024 julia> length(typeof(Core.kwcall).name.backedges) 196 julia> length(typeof(convert).name.backedges) 1510 ``` (cherry picked from commit 1c26f43)
When this API was added, this function inlined, which is important, because the API relies on the allocation of the `Ref` being elided. At some point (I went back to 1.8) this regressed. For example, it is currently responsible for substantially all non-Expr allocations in JuliaParser. Before (parsing all of Base with JuliaParser): ``` │ Memory estimate: 76.93 MiB, allocs estimate: 719922. ``` After: ``` │ Memory estimate: 53.31 MiB, allocs estimate: 156. ``` Also add a test to make sure this doesn't regress again. (cherry picked from commit d6294ba)
(cherry picked from commit 8567a3a)
|
Gonna try the packages timing out in isolation @nanosoldier |
|
The package evaluation job you requested has completed - possible new issues were detected. Report summary✖ Packages that failed19 packages failed only on the current version.
1 packages failed on the previous version too. ✔ Packages that passed tests11 packages passed tests on the previous version too. ➖ Packages that were skipped altogether1 packages were skipped only on the current version.
1 packages were skipped on the previous version too. |
Manual backport of #58586 . Co-authored-by: Sukera <11753998+Seelengrab@users.noreply.github.com>
… to 6cc0405 (#58725) Stdlib: LinearAlgebra URL: https://github.com/JuliaLang/LinearAlgebra.jl.git Stdlib branch: release-1.12 Julia branch: backports-release-1.12 Old commit: 7264a49 New commit: 6cc0405 Julia version: 1.12.0-beta4 LinearAlgebra version: 1.12.0 Bump invoked by: @jishnub Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaLang/LinearAlgebra.jl@7264a49...6cc0405 ``` $ git log --oneline 7264a49..6cc0405 6cc0405 Backports release 1.12 (#1379) 73ff52d Change default symmetriceigen algorithm back to RobustRepresentations (#1363) 7194038 clarify eigen docstring (#1364) ``` Co-authored-by: jishnub <10461665+jishnub@users.noreply.github.com>
…to cdbad55 (#58724) Stdlib: SparseArrays URL: https://github.com/JuliaSparse/SparseArrays.jl.git Stdlib branch: release-1.12 Julia branch: backports-release-1.12 Old commit: 72c7cac New commit: cdbad55 Julia version: 1.12.0-beta4 SparseArrays version: 1.12.0 Bump invoked by: @fredrikekre Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaSparse/SparseArrays.jl@72c7cac...cdbad55 ``` $ git log --oneline 72c7cac..cdbad55 cdbad55 Delete lock interface implementation from UmfpackLU (#617) 415bc9e [release-1.12] Run CI with Julia 1.12 (#635) d921308 fix libsuitesparseconfig bug f51dace Fix compat for SuiteSparse_jll fa9736c [release-1.12] Run CI with Julia 1.12 d1b0cd0 Backports for v1.12 (#624) e1817e8 Clarify pros, cons and limitations of Cholesky and LDLt (#621) b38f273 Use `libsuitesparseconfig` from JLL (#620) 66d65cc Relax `eltype` in `Diagonal` `ldiv!`/`rdiv!` (#616) 2ae8768 `ldiv!` for `Diagonal` and a sparse vector (#613) 08a15ca Replace `v == zero(v)` with `_iszero` (#610) 7ec2889 Fix `issymmetric` for matrices with empty columns (#606) f3610c0 SuiteSparse 7.10.1 wrappers and simpliefied wrapper generator process (#608) 9548149 Use `axes` instead of `1:size` (#602) ae727fe Explicitly import `Matrix` and `Vector` in CHOLMOD (#601) ce852af cleanup support for Float32 and ComplexF32 (#597) ``` Co-authored-by: fredrikekre <11698744+fredrikekre@users.noreply.github.com>
hmm |
This method intends to be a SIMD-able optimization for reductions with `min` and `max`, but it fails to achieve those goals on nearly every architecture. For example, on my Mac M1 the generic implementation is more than **6x** faster than this method. Fixes #45932, fixes #36412, fixes #36081. --------- Co-authored-by: Mosè Giordano <765740+giordano@users.noreply.github.com> (cherry picked from commit 99ba3c7)
Co-authored-by: Jameson Nash <vtjnash@gmail.com> (cherry picked from commit 3ed13ea)
Currently, `similar(::CodeUnits)` works as expected by going through the
generic `AbstractArray` method. However, the fallback method hit by
`similar(::Type{<:CodeUnits}, dims)` does not work, as it assumes the
existence of a constructor that accepts an `UndefInitializer`. This can
be made to work by defining a corresponding `similar` method that
returns an `Array`.
One could make a case that this is a bugfix since it was arguably a bug
that this method didn't work given that `CodeUnits` is an
`AbstractArray` subtype and the other `similar` methods work. If anybody
buys that argument, it could be nice to backport this; it came up in
some internal code that uses Arrow.jl and JSON3.jl together.
(cherry picked from commit 8e524c7)
Use an atomic fetch and add to fix a data race in `Module()` identified by tsan: ``` ./usr/bin/julia -t4,0 --gcthreads=1 -e 'Threads.@threads for i=1:100 Module() end' ================== WARNING: ThreadSanitizer: data race (pid=5575) Write of size 4 at 0xffff9bf9bd28 by thread T9: #0 jl_new_module__ /home/user/c/julia/src/module.c:487:22 (libjulia-internal.so.1.13+0x897d4) #1 jl_new_module_ /home/user/c/julia/src/module.c:527:22 (libjulia-internal.so.1.13+0x897d4) #2 jl_f_new_module /home/user/c/julia/src/module.c:649:22 (libjulia-internal.so.1.13+0x8a968) #3 <null> <null> (0xffff76a21164) #4 <null> <null> (0xffff76a1f074) #5 <null> <null> (0xffff76a1f0c4) #6 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5ea04) #7 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5ea04) #8 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9e4c4) #9 start_task /home/user/c/julia/src/task.c:1249:19 (libjulia-internal.so.1.13+0x9e4c4) Previous write of size 4 at 0xffff9bf9bd28 by thread T10: #0 jl_new_module__ /home/user/c/julia/src/module.c:487:22 (libjulia-internal.so.1.13+0x897d4) #1 jl_new_module_ /home/user/c/julia/src/module.c:527:22 (libjulia-internal.so.1.13+0x897d4) #2 jl_f_new_module /home/user/c/julia/src/module.c:649:22 (libjulia-internal.so.1.13+0x8a968) #3 <null> <null> (0xffff76a21164) #4 <null> <null> (0xffff76a1f074) #5 <null> <null> (0xffff76a1f0c4) #6 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5ea04) #7 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5ea04) #8 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9e4c4) #9 start_task /home/user/c/julia/src/task.c:1249:19 (libjulia-internal.so.1.13+0x9e4c4) Location is global 'jl_new_module__.mcounter' of size 4 at 0xffff9bf9bd28 (libjulia-internal.so.1.13+0x3dbd28) ``` (cherry picked from commit 9039555)
would fix #57170, fix #54623 @nanosoldier `runbenchmarks("array", vs=":master")` (cherry picked from commit e853a4f)
Update the stdlib OpenSSL to 3.5.1. This is a candidate for backporting to Julia 1.12 if there is another beta release. (cherry picked from commit 41570e9)
(cherry picked from commit c0cc1e1)
KristofferC
force-pushed
the
backports-release-1.12
branch
from
94d95af to
5d106c5
Compare
Currently, this hits a fallback method that assumes that division is defined for the elements of the range. After this, the following works: ```julia julia> r = StepRangeLen(CartesianIndex(1), CartesianIndex(1), 3); julia> r[1] in r true julia> CartesianIndex(0) in r false ``` (cherry picked from commit 24b3273)
KristofferC
force-pushed
the
backports-release-1.12
branch
from
5d106c5 to
304847e
Compare
This adds a test for an invalidation failure when `consts` are redefined. The failure occurs only in "edge" mode, i.e., due to the separation of precompilation from the user's running session. (cherry picked from commit c13ea2b)
Currently we log the invalidated backedges of a binding invalidation, but the actual trigger of the invalidation is not logged. This is needed to allow SnoopCompile to attribute a cause to those invalidations. (cherry picked from commit 75946ce)
…orld (#58830) There are three categories of methods we need to worry about during staticdata validation: 1. New methods added to existing generic functions 2. New methods added to new generic functions 3. Existing methods that now have new CodeInstances In each of these cases, we need to check whether any of the implicit binding edges from the method's source was invalidated. Currently, we handle this for 1 and 2 by explicitly scanning the method on load. However, we were not tracking it for case 3. Fix that by using an extra bit in did_scan_method that gets set when we see an existing method getting invalidated, so we know that we need to drop the corresponding CodeInstances during load. Fixes #58346 (cherry picked from commit 77b90b9)
Add the same optimization from Method to MethodInstance, although the performance gain seems to be negligible in my specific testing, there doesn't seem any likely downside to adding one caching bit to avoid some recomputations. (cherry picked from commit b459d88)
This is a bugfix and code cleanup, since it wasn't properly encoding and checking for newly "missing"-type backedges `fully_covering` during verification. Co-authored-by: Claude <noreply@anthropic.com> (cherry picked from commit 6efd218)
(cherry picked from commit c3282ce)
This reverts commit 304847e.
Seems to only be on mac... |
|
Ah, maybe fixed by #58643 |
Replace fragile timing-based synchronization with proper condition variable signaling to ensure PATH cache updates complete before testing completions. This eliminates test flakiness on systems where the cache update takes longer than 5s. The test failure was seen in CI: https://buildkite.com/julialang/julia-master/builds/48273 (cherry picked from commit a4ab110)
…o 3679026 (#58910) Stdlib: Distributed URL: https://github.com/JuliaLang/Distributed.jl Stdlib branch: master Julia branch: backports-release-1.12 Old commit: 51e5297 New commit: 3679026 Julia version: 1.12.0-beta4 Distributed version: 1.11.0(Does not match) Bump invoked by: @DilumAluthge Powered by: [BumpStdlibs.jl](https://github.com/JuliaLang/BumpStdlibs.jl) Diff: JuliaLang/Distributed.jl@51e5297...3679026 ``` $ git log --oneline 51e5297..3679026 3679026 Merge pull request #137 from JuliaLang/dpa/dont-use-link-local 875cd5a Rewrite the code to be a bit more explicit 2a6ee53 Non-link-local IP4 > non-link-local IP6 > link-local IP4 > link-local IP6 c0e9eb4 Factor functionality out into separate `choose_bind_addr()` function 86cbb8a Add explanation 0b7288c Worker: Bind to the first non-link-local IPv4 address ff8689a Merge pull request #131 from JuliaLang/spawnat-docs ba3c843 Document that `@spawnat :any` doesn't do load-balancing ``` Co-authored-by: DilumAluthge <5619885+DilumAluthge@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backported PRs:
minimum/maximummethod #58267jl_get_global#58540FieldError#58507reshapeforArray#58672inforCartesianIndexranges #58616isdefined_tfuncaccuracy forMustAlias#58743CartesianIndicesiteration #58742trim_modeparameter to JIT type-inference entrypoint #58817(un)preserve_handleas@nospecialize#58844similarmethod forType{<:CodeUnits}#57826hvncatdimension calculation to fix issue with 0-length elements in first dimension #58881to_power_typetodeprecated.jlsince some packages call it #58886Need manual backport:
cfunction#58722Contains multiple commits, manual intervention needed:
Non-merged PRs with backport label:
--trace-compile#58535transcode: prevent Windows sysimage invalidation #58038@nospecializeforstring_index_err#57604maxthreadidfromThreads#57490