[enhancement] enable array_api return values from from_table
#2441
Conversation
|
/intelci: run |
icfaust
changed the title
from_tablefrom_table
|
/intelci: run |
|
|
||
| Parameters | ||
| ---------- | ||
| *args : {scalar, numpy array, sycl_usm_ndarray, csr_matrix, or csr_array} | ||
| *args : scalar, numpy array, sycl_usm_ndarray, array API standard array, |
There was a problem hiding this comment.
Is sycl_usm_ndarray this specific class? https://intelpython.github.io/dpctl/latest/api_reference/dpctl/generated/dpctl.tensor.usm_ndarray.html
| Returns | ||
| ------- | ||
| tables: {oneDAL homogeneous tables} | ||
| tables: oneDAL homogeneous tables |
There was a problem hiding this comment.
Format here would need to be:
name_of_returned_variable : type_of_returned variable
Description about it.
|
Fixes must be made to spmd testing :( |
|
This has implications about array-api-compat. I need to rethink this strategy. |
|
The situation is as follows. I didn't look thoroughly enough into the reasons behind array-api-compat and its inclusion into sklearn. Even if we fully support array_api inputs, it will not mean we will support pytorch out of the box. Pytorch support via array api comes via getting the namespace generating from array-api-compat, meaning manually searching for array_namespace won't work (and the reason why sklearn has |
|
I'm going to have to modify the ingestion scripts ( |
Description
** Dependent on #2430 **
leaning into the functional style development for
from_table, its is generalized to work with expected major frameworks (dpctl, dpnp, and various flavors of array_api arrays). This is done by changing thefrom_tableAPI to depend not on kwargssua_ifaceandsycl_queuebut on a single keywordlike, which can either be an array type or a callable which will return the expected array type. The returned array will preserve the aspects of the array (shape, type, row or column major contiguous, etc.) and in most circumstances the device. A major exception is sycl_usm cpu USM allocations, which require a manual memcpy to a CPU allocation.Follow-on work must consistently apply the use of
likeas the roll-out of raw_inputs in #2153, as some estimators will force values to the return type and others not. As this is inconsistently applied, it must be fixed for any array_api-zero copy enabled estimator.Changes:
__sycl_usm_array_interface__to throw an AttributeError if a queue is unavailable, rather than set to None. The attribute should only be available when a USM allocation is made, which is not the case for CPU data out of oneDAL. This check is placed early in the attribute code so that it can be used for checking for sycl usm conversion withhas_attrreturn_type_constructorfunction to generalize oneDAL table to array construction_data_conversion.pysua_ifaceandsycl_queuekeywords to uselikereturn_type_constructorNOTES: The initial support version for
from_tableis not general enough and highly coded for sycl usm inputs. This changes the interface to mirror the type (and possibly queue) of the data. Getting this integrated will begin the fight against the technical debt introduced withuse_raw_inputs. The core of this PR is simple, the changes needed associated with the technical debt will be brutal.Incremental algorithms will first collect and store the return type generator function via the
return_type_constructorfunction. This is similar to how the queue must be stored, which will be applied at the end at the finalization step. The serialization deserialization of this isn't possible, so it should be handled similarly as the queue.If we want to type hint these changes, I need help. I'm not sure what to do for the specifics of stating an array type.
I don't think performance benchmarks are necessary, but can do upon request.
PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.
You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.
For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).
Checklist to comply with before moving PR from draft:
PR completeness and readability
Testing