-
Notifications
You must be signed in to change notification settings - Fork 13
ENH: add quantile
#341
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
betatim
wants to merge
22
commits into
data-apis:main
Choose a base branch
from
betatim:add-quantile
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
ENH: add quantile
#341
Changes from 12 commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
f7cac01
Add delegation for `quantile`
betatim 9577f11
Formatting
betatim e8a7d30
Fix scipy version
betatim 10b9ec8
Formatting
betatim 98570f6
Remove superfluous comments
betatim 470f8b4
Remove unsupported method
betatim a2eefa0
More noqa
betatim 37acd5b
yet more noqa
betatim c3501e8
Move quantile implementation to new file
betatim 7a7934c
Remove duplicated code
betatim 3095889
docstring keepdims
lucascolley bd55318
lint
lucascolley 1ef7d5e
improve style
lucascolley 440106f
fix list
lucascolley 0f28550
Merge branch 'main' into add-quantile
lucascolley 3ef6727
Raise exception for invalid q values
betatim 13a5507
Merge remote-tracking branch 'origin/add-quantile' into add-quantile
betatim 007a61f
Tweak
betatim 1ccdac4
noqa
betatim ebcec0e
More lint pleasure
betatim 5c974a4
Delegate to dask directly
betatim 477c916
Fix
betatim File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -19,6 +19,7 @@ | |
nunique | ||
one_hot | ||
pad | ||
quantile | ||
setdiff1d | ||
sinc | ||
``` |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -5,6 +5,7 @@ | |
from typing import Literal | ||
|
||
from ._lib import _funcs | ||
from ._lib._quantile import quantile as _quantile | ||
from ._lib._utils._compat import ( | ||
array_namespace, | ||
is_cupy_namespace, | ||
|
@@ -18,7 +19,7 @@ | |
from ._lib._utils._helpers import asarrays | ||
from ._lib._utils._typing import Array, DType | ||
|
||
__all__ = ["isclose", "one_hot", "pad"] | ||
__all__ = ["isclose", "one_hot", "pad", "quantile"] | ||
|
||
|
||
def isclose( | ||
|
@@ -247,3 +248,95 @@ def pad( | |
return xp.nn.functional.pad(x, tuple(pad_width), value=constant_values) # type: ignore[arg-type] # pyright: ignore[reportArgumentType] | ||
|
||
return _funcs.pad(x, pad_width, constant_values=constant_values, xp=xp) | ||
|
||
|
||
def quantile( | ||
x: Array, | ||
q: Array | float, | ||
/, | ||
*, | ||
axis: int | None = None, | ||
keepdims: bool | None = None, | ||
method: str = "linear", | ||
xp: ModuleType | None = None, | ||
) -> Array: | ||
""" | ||
Compute the q-th quantile(s) of the data along the specified axis. | ||
|
||
Parameters | ||
---------- | ||
x : array of real numbers | ||
Data array. | ||
q : array of float | ||
Probability or sequence of probabilities of the quantiles to compute. | ||
Values must be between 0 and 1 (inclusive). Must have length 1 along | ||
`axis` unless ``keepdims=True``. | ||
axis : int or None, default: None | ||
Axis along which the quantiles are computed. ``None`` ravels both `x` | ||
and `q` before performing the calculation. | ||
keepdims : bool or None, default: None | ||
By default, the axis will be reduced away if possible | ||
(i.e. if there is exactly one element of `q` per axis-slice of `x`). | ||
If `keepdims` is set to True, the axes which are reduced are left in the | ||
result as dimensions with size one. With this option, the result will | ||
broadcast correctly against the original array `x`. | ||
If `keepdims` is set to False, the axis will be reduced away if possible, | ||
and an error will be raised otherwise. | ||
method : str, default: 'linear' | ||
The method to use for estimating the quantile. The available options are: | ||
'inverted_cdf', 'averaged_inverted_cdf', 'closest_observation', | ||
'interpolated_inverted_cdf', 'hazen', 'weibull', 'linear' (default), | ||
'median_unbiased', 'normal_unbiased'. | ||
xp : array_namespace, optional | ||
The standard-compatible namespace for `x` and `q`. Default: infer. | ||
|
||
Returns | ||
------- | ||
array | ||
An array with the quantiles of the data. | ||
|
||
Examples | ||
-------- | ||
>>> import array_api_strict as xp | ||
>>> import array_api_extra as xpx | ||
>>> x = xp.asarray([[10, 8, 7, 5, 4], [0, 1, 2, 3, 5]]) | ||
>>> xpx.quantile(x, 0.5, axis=-1) | ||
Array([7., 2.], dtype=array_api_strict.float64) | ||
>>> xpx.quantile(x, [0.25, 0.75], axis=-1) | ||
Array([[5., 8.], | ||
[1., 3.]], dtype=array_api_strict.float64) | ||
""" | ||
# We only support a subset of the methods supported by scipy.stats.quantile. | ||
# So we need to perform the validation here. | ||
methods = { | ||
"inverted_cdf", | ||
"averaged_inverted_cdf", | ||
"closest_observation", | ||
"hazen", | ||
"interpolated_inverted_cdf", | ||
"linear", | ||
"median_unbiased", | ||
"normal_unbiased", | ||
"weibull", | ||
} | ||
if method not in methods: | ||
raise ValueError(f"`method` must be one of {methods}") # noqa: EM102 | ||
|
||
xp = array_namespace(x, q) if xp is None else xp | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. scikit-learn/scikit-learn#31671 (comment) suggests that delegation to some existing array libraries may be desirable here |
||
try: | ||
import scipy # type: ignore[import-untyped] | ||
from packaging import version | ||
|
||
# The quantile function in scipy 1.16 supports array API directly, no need | ||
# to delegate | ||
if version.parse(scipy.__version__) >= version.parse("1.16"): # pyright: ignore[reportUnknownArgumentType] | ||
from scipy.stats import ( # type: ignore[import-untyped] | ||
quantile as scipy_quantile, | ||
) | ||
|
||
return scipy_quantile(x, p=q, axis=axis, keepdims=keepdims, method=method) | ||
except (ImportError, AttributeError): | ||
pass | ||
|
||
return _quantile(x, q, axis=axis, keepdims=keepdims, method=method, xp=xp) | ||
lucascolley marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.