Skip to content

Speed up permsort by utilizing stability of the default sorting algorithm #47587

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 41 commits into
base: master
Choose a base branch
from
Draft

Speed up permsort by utilizing stability of the default sorting algorithm #47587

Changes from 3 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
42c70a1
initial functionality
Oct 17, 2022
61e4006
support 5- and 3-argument sort! for backwards compatability
Oct 29, 2022
901182c
test for bug that slipped through test suite
Oct 17, 2022
e032ba6
fix bug
Oct 17, 2022
e6cfee0
make send_to_end more human friendly (and less compiler friendly! int…
Oct 20, 2022
f160582
Give each sorting pass and DEFAULT_STABLE a docstring
Oct 29, 2022
15a4484
add tests and fix typos they unveiled
Oct 30, 2022
d82b090
avoid potential name conflict
Nov 1, 2022
029cbae
switch to custom keyword handling
Nov 1, 2022
d3bdca3
remove InsertionSortAlg and MergeSortAlg
Nov 2, 2022
2232cac
better algorithm display
Nov 2, 2022
a574c7f
stop passing U around
Nov 2, 2022
05de36e
remove lenm1
Nov 6, 2022
70290d6
fix unexpected allocations in Radix Sort
Nov 7, 2022
f06de10
fix doctests? I have no idea how
Nov 7, 2022
38f4512
support and test backwards compatability with packages that depend in…
Nov 9, 2022
383b9d2
Merge branch 'master' into sort-dispatch
Nov 9, 2022
d8ae968
improve extensibility tests
Nov 10, 2022
c633419
overhall scratch space handling
Nov 11, 2022
32a6f54
Merge branch 'master' into sort-dispatch
LilithHafner Nov 14, 2022
a2c2646
Consistency with other constructors
Nov 15, 2022
71e8fa1
Introduce PermUnstable to speed up sortperm
petvana Nov 15, 2022
812c917
Fix a mistake
petvana Nov 15, 2022
e752ea7
pass around even fewer easily computed things in kw to reduce load on…
Nov 18, 2022
15666f2
Merge branch 'master' into sort-dispatch
LilithHafner Nov 18, 2022
04399d9
Merge branch 'sort-dispatch' into pv/PermUnstable-v4
petvana Nov 18, 2022
34621c7
Merge branch 'pv/PermUnstable-v4' into pv/PermUnstable-v5
petvana Dec 5, 2022
7e6f103
Restore the PR
petvana Dec 5, 2022
77b2b08
Rename to PermFast
petvana Dec 5, 2022
36d3ff3
Introduce send_to_end_stable!
petvana Dec 5, 2022
1fe68d9
Fix spacing
petvana Dec 5, 2022
dd1d89b
Fix trailing whitespace
petvana Dec 5, 2022
91c2d2a
Merge branch 'master' into pv/PermUnstable-v4
petvana Dec 5, 2022
20ddeb4
Small fixes
petvana Dec 6, 2022
c14432b
Fix sortperm!
petvana Dec 6, 2022
176d779
Merge branch 'master' into pv/PermUnstable-v4
petvana Dec 13, 2022
a2f9710
Commit suggestion from review
petvana Dec 14, 2022
2d2cf4d
De-duplicate code
petvana Dec 14, 2022
ef8e8eb
Merge branch 'pv/PermUnstable-v4' of github.com:petvana/julia into pv…
petvana Dec 14, 2022
3b972eb
Re-use scratch array
petvana Dec 14, 2022
9b5be34
Merge branch 'master' into pv/PermUnstable-v4
petvana Dec 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 12 additions & 15 deletions base/sort.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -603,7 +603,7 @@ function _sort!(v::AbstractVector, a::MissingOptimization, o::Ordering, kw)
_sort!(WithoutMissingVector(v, unsafe=true), a.next, o, (;kw..., lo, hi))
elseif eltype(v) <: Integer && (o isa Perm{DirectOrdering} || o isa PermFast{DirectOrdering}) &&
nonmissingtype(eltype(o.data)) != eltype(o.data)
PermT = o isa Perm{DirectOrdering} ? Perm : PermFast
PermT = o isa PermFast ? PermFast : Perm
lo, hi = send_to_end!(i -> ismissing(@inbounds o.data[i]), v, o)
_sort!(v, a.next, PermT(o.order, WithoutMissingVector(o.data, unsafe=true)), (;kw..., lo, hi))
else
Expand Down Expand Up @@ -644,25 +644,22 @@ function _sort!(v::AbstractVector, a::IEEEFloatOptimization, o::Ordering, kw)
else
_sort!(iv, a.next, Forward, (;kw..., lo=j+1, hi, scratch))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LilithHafner Btw, is scratch type-stable here, i.e., can compiler infer that scratch cannot be Nothing?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recall that removing the if statement entirely introduced dynamic dispatch. IICU, all of type inference is an implementation detail, so I'm not totally sure, but I believe that we need an if statement to force union splitting though it works well whether we use _sort!(..., nothing) or _sort!(..., scratch) because the compiler can determine the type of scratch at compile time..

end
elseif eltype(v) <: Integer && o isa Perm && o.order isa DirectOrdering && is_concrete_IEEEFloat(eltype(o.data))
lo, hi = send_to_end!(i -> isnan(@inbounds o.data[i]), v, o.order, true; lo, hi)
ip = reinterpret(UIntType(eltype(o.data)), o.data)
j = send_to_end!(i -> after_zero(o.order, @inbounds o.data[i]), v; lo, hi)
scratch = _sort!(v, a.next, Perm(Reverse, ip), (;kw..., lo, hi=j))
if scratch === nothing # Union split
_sort!(v, a.next, Perm(Forward, ip), (;kw..., lo=j+1, hi, nothing))
elseif eltype(v) <: Integer && (o isa Perm || o isa PermFast) && o.order isa DirectOrdering && is_concrete_IEEEFloat(eltype(o.data))
if o isa Perm
lo, hi = send_to_end!(i -> isnan(@inbounds o.data[i]), v, o.order, true; lo, hi)
j = send_to_end!(i -> after_zero(o.order, @inbounds o.data[i]), v; lo, hi)
PermT = Perm
else
_sort!(v, a.next, Perm(Forward, ip), (;kw..., lo=j+1, hi, scratch))
lo, hi = send_to_end_stable!(i -> isnan(@inbounds o.data[i]), v, o.order; lo, hi)
j = send_to_end_stable!(i -> after_zero(o.order, @inbounds o.data[i]), v; lo, hi)
PermT = PermFast
end
elseif eltype(v) <: Integer && o isa PermFast && o.order isa DirectOrdering && is_concrete_IEEEFloat(eltype(o.data))
lo, hi = send_to_end_stable!(i -> isnan(@inbounds o.data[i]), v, o.order; lo, hi)
ip = reinterpret(UIntType(eltype(o.data)), o.data)
j = send_to_end_stable!(i -> after_zero(o.order, @inbounds o.data[i]), v; lo, hi)
scratch = _sort!(v, a.next, PermFast(Reverse, ip), (;kw..., lo, hi=j))
scratch = _sort!(v, a.next, Perm(Reverse, ip), (;kw..., lo, hi=j))
if scratch === nothing # Union split
_sort!(v, a.next, PermFast(Forward, ip), (;kw..., lo=j+1, hi, nothing))
_sort!(v, a.next, PermT(Forward, ip), (;kw..., lo=j+1, hi, nothing))
else
_sort!(v, a.next, PermFast(Forward, ip), (;kw..., lo=j+1, hi, scratch))
_sort!(v, a.next, PermT(Forward, ip), (;kw..., lo=j+1, hi, scratch))
end
else
_sort!(v, a.next, o, kw)
Expand Down