Skip to content

ugly special-casing of AbstractQ for concatenation (cat) #1233

Open
@nsajko

Description

@nsajko

This is the code in question, introduced in JuliaLang/julia#51132:

# used in concatenations: Base.__cat_offset1!
Base._copy_or_fill!(A, inds, Q::AbstractQ) = (A[inds...] = collect(Q))
# overloads of helper functions
Base.cat_size(A::AbstractQ) = size(A)
Base.cat_size(A::AbstractQ, d) = size(A, d)
Base.cat_length(a::AbstractQ) = prod(size(a))
Base.cat_ndims(a::AbstractQ) = ndims(a)
Base.cat_indices(A::AbstractQ, d) = axes(A, d)
Base.cat_similar(A::AbstractQ, T::Type, shape::Tuple) = Array{T}(undef, shape)
Base.cat_similar(A::AbstractQ, T::Type, shape::Vector) = Array{T}(undef, shape...)

# used in concatenations: Base.__cat_offset1!
Base._copy_or_fill!(A, inds, Q::AbstractQ) = (A[inds...] = collect(Q))
# overloads of helper functions
Base.cat_size(A::AbstractQ) = size(A)
Base.cat_size(A::AbstractQ, d) = size(A, d)
Base.cat_length(a::AbstractQ) = prod(size(a))
Base.cat_ndims(a::AbstractQ) = ndims(a)
Base.cat_indices(A::AbstractQ, d) = axes(A, d)
Base.cat_similar(A::AbstractQ, T::Type, shape::Tuple) = Array{T}(undef, shape)
Base.cat_similar(A::AbstractQ, T::Type, shape::Vector) = Array{T}(undef, shape...)

All of the above functions getting AbstractQ-specific methods are internal, implementation details of Base. All of them only have methods for (generic) AbstractArray and AbstractQ (except cat_similar, which also has a special implementation for Array).

Historical note: AbstractQ used to subtype AbstractArray, but this was changed, with the above methods presumably being added only for compatibility with the previous behavior, back when AbstractQ <: AbstractArray.

This situation, with private Base methods being extended only from LinearAlgebra (and only by AbstractQ):

  • Seems like a possible maintenance problem in view of the possibility of LinearAlgebra becoming just a regular upgradable package, given that the functions are not part of the public interface of Julia
  • Is not fair to the ecosystem in general, because:
    • It's hypocritical to forbid user packages from accessing implementation details but then do it from a stdlib.
    • Some of these functions have low-enough method counts for the world-splitting optimization to kick in. This means that concatenation of values not at all related to linear algebra may cause AbstractQ-related code to be compiled.

Perhaps these methods could simply be dropped, as the PR that introduced them says they're unused? Otherwise, I think some of these options should be considered:

  • Lower the world-splitting threshold for these functions (Base.Experimental.@max_methods)
  • Replace these functions by a public API extendable by any package, not just LinearAlgebra
  • If possible, make the concatenation logic in Base general enough for these methods to become unnecessary, so they could be dropped from LinearAlgebra without affecting functionality.

Metadata

Metadata

Assignees

No one assigned

    Labels

    arrays[a, r, r, a, y, s]

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions