|
| 1 | +--- |
| 2 | +title: "Summit Follow Up June 17" |
| 3 | +--- |
| 4 | + |
| 5 | +# Friday June 17 2023, 14:00 to 15:00 UTC |
| 6 | + |
| 7 | +## Attendees |
| 8 | + |
| 9 | +- Dan |
| 10 | +- Erik |
| 11 | +- Isaac |
| 12 | +- CJ |
| 13 | +- Julien |
| 14 | + |
| 15 | +## Agenda |
| 16 | + |
| 17 | +- Sort out and attribute [what remains to be done](https://hackmd.io/1Q2832LDR_2Uv_-cV-wnYg#What-remains-to-be-done) |
| 18 | +- Roadmap/ plan for array-like sparse arrays |
| 19 | + - Deprecation plan for matrices |
| 20 | + - [Creation routines](https://github.com/scipy/scipy/issues/18592) |
| 21 | + - Interop with other sparse matrix packages (isaac) |
| 22 | +- Blog post |
| 23 | + |
| 24 | +## Notes |
| 25 | + |
| 26 | +- Introduction, Erik works on libraries using sparse datastructures |
| 27 | + - [Julien] Erik: here is [the worklog of the summit](https://hackmd.io/iEtdfbxfSbGwOAJTXmqyIQ) |
| 28 | + |
| 29 | +### What remains to be done |
| 30 | + |
| 31 | +- How do we work |
| 32 | + - [Dan] Where does in-progress work go |
| 33 | + - CJ's fork, make PRs onto branch there |
| 34 | + - Using the view on the diff on GitHub might prevent the creation of spurious Draft PR |
| 35 | + - Dense pretending to be sparse can be ephermeral, never needs to go to scipy |
| 36 | + - Dan will make a branch for exploration that people can fork and make PRs to. Dan will try to keep it uptodate with the main branch too. |
| 37 | +- 1d sparse array support |
| 38 | + Timeline? - Dan + CJ working on it - Targetting 1.12 |
| 39 | +- nd sparse array support |
| 40 | + - Julien, CJ: seems like low priority and this would be a huge maintainance cost SciPy for relatively rare use-cases |
| 41 | + - CJ: Array api makes interop easier, we can lean on external packages for now |
| 42 | +- creation functions [scipy#18592](https://github.com/scipy/scipy/issues/18592) |
| 43 | + - Want to get them out in a single release |
| 44 | + - Is Ross going to implement these, or need to assign elsewhere? |
| 45 | + - Current plan: `_array` methods, so we can apply this to the fast_matrix_market io at the same type |
| 46 | + - Bundling in api changes (like tuples for shapes) |
| 47 | +- Deprecate `isspmatrix_` methods? |
| 48 | + - Sounds like combination of classes + format attribute cover this |
| 49 | + - (Julien) Deprecation plan: have suggested code to replace these things with |
| 50 | + - Add a section in the documentation explaining the change of semantic and a migration plan for downstream libraries during the deprecation cycle. |
| 51 | + - Make a roadmap for array api |
| 52 | +- scikit-learn |
| 53 | + - When will scikit-learn be willing to support a compatible version of scipy? |
| 54 | + - Julien: |
| 55 | + - reasoning is based on some linux package manager not supporting latest numpy scipy |
| 56 | + - Personally would like scikit-learn to follow SPEC 0 |
| 57 | + - See discussion for [1.3](https://github.com/scikit-learn/scikit-learn/issues/26438) |
| 58 | + - [Dedicated RFC](https://github.com/scikit-learn/scikit-learn/issues/26418) |
| 59 | + - Some part of scikit-learn's behavior depends on SciPy's version (e.g. see [this one](https://github.com/scikit-learn/scikit-learn/blob/784ba9ef9f65d5e4e33087dd7f5b87d65b605efc/sklearn/preprocessing/_polynomial.py#L61-L73)); we might potentialy rely on similar mechanism for the support of sparse arrays (e.g. reshaping outputs) |
| 60 | + - scikit-learn uses nightly builds to test the developer version of SciPy and NumPy and failing tests are updated in [this issue](https://github.com/scikit-learn/scikit-learn/issues/26154). We can watch those for breakage in scikit-learn when we make changes in SciPy. |
| 61 | +- General speed ups |
| 62 | + - "Fast path for canonical" |
| 63 | + - TODO (Isaac) |
| 64 | + - Figure out canonical for all the formats, probably normalize APIs |
| 65 | + - Add keyword public argument for constructors |
| 66 | + |
| 67 | +## Broad plan for deprecation |
| 68 | + |
| 69 | +- Add support for sparse_arrays everywhere we can |
| 70 | + - Creation functions: target 1.12 |
| 71 | + - 1d COO: target 1.12 |
| 72 | +- Deprecate spmatrix-specific functions (where there's an easy replacement) |
| 73 | + - Target: 1.12 |
| 74 | +- Once everything works with sparse arrays, then deprecate spmatrix |
| 75 | + - And the sparse array API is stable! (specifically, 1d support) |
| 76 | + - Target: 1.13 |
| 77 | + |
| 78 | +## Actionable items |
| 79 | + |
| 80 | +- [Julien] Add a section in the documentation explaining the change of semantic and a migration plan for downstream libraries during the deprecation cycle. |
| 81 | +- [Julien] Drop `isspmatrix_-` checks in scikit-learn and use `issparse` and `format` |
| 82 | + |
| 83 | +## Next meeting date and time |
| 84 | + |
| 85 | +- Every two weeks with the option to drop every other meeting (at least once a month) |
| 86 | + > [name=Julien] +1. I proposed to meet every two week for 1h on the same timeslot |
0 commit comments