Skip to content

dot overflows #15690

Open
JuliaLang/LinearAlgebra.jl
#1400
@timholy

Description

@timholy

Filing for future reference (would be too much of an interruption to fix now):

julia> a = rand(UInt8, 5)
5-element Array{UInt8,1}:
 0x08
 0xcd
 0x4a
 0x5d
 0x1f

julia> b = rand(UInt8, 5)
5-element Array{UInt8,1}:
 0x7c
 0xf4
 0x2c
 0x86
 0xef

julia> dot(a, b)
0x9b

The fix is less trivial than it seems: the r_promote infrastructure in reduce.jl seems promising, but it doesn't handle the product of elements, only the sum of the accumulator. We might need a widen_op analogous to promote_op to prevent these kinds of overflows.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions