Version 1.9.0 #1664

jpivarski · 2022-09-02T19:29:53Z

jpivarski
Sep 2, 2022
Maintainer

These are differences from 1.8.0 to 1.9.0. Most only affect Awkward Array version 2, in the awkward._v2 submodule of 1.9.0. The exceptions are called out with "v1" in the subject line.

Features

feat: C++ refactoring: ak.to_pandas by @ioanaif in C++ refactoring: ak.to_pandas #1369
feat: C++ refactoring: ak.nan_to_num by @ioanaif in C++ refactoring: ak.nan_to_num #1352
feat: C++ refactoring: ak.run_lengths by @ioanaif in C++ refactoring: ak.run_lengths #1347
feat: add is_tuple describe operation by @agoose77 in Feat: add is_tuple describe operation #1351
feat: C++ refactoring: ak.unzip by @ioanaif in C++ refactoring: ak.unzip #1354
feat: pure Cling demo and improvements to C++ JIT infrastructure by @jpivarski in Pure Cling demo and improvements to C++ JIT infrastructure #1359
feat: C++ refactoring: ak.broadcast_arrays by @ioanaif in C++ refactoring: ak.broadcast_arrays #1368
feat: irst version of ak._v2.from_parquet by @jpivarski in First version of ak._v2.from_parquet #1338
feat: C++ refactoring: ak.copy by @ioanaif in C++ refactoring: ak.copy #1367
feat: C++ refactoring: ak.unflatten by @ioanaif in C++ refactoring: ak.unflatten #1360
feat: add depth_limit to ak.broadcast_arrays by @agoose77 in Feat: add depth_limit to ak.broadcast_arrays #1373
feat: Allow NumPy arrays in CppStatements; fix row_groups in single-file from_parquet. by @jpivarski in Allow NumPy arrays in CppStatements; fix row_groups in single-file from_parquet. #1376
feat: This PR sets up the architecture to call CuPy Raw Kernels from Awkward. by @swishdiff in This PR sets up the architecture to call CuPy Raw Kernels from Awkward. #1355
feat: Reducers with axis=None and typetracers. by @jpivarski in Reducers with axis=None and typetracers. #1380
feat: This PR adds the generated kernels and simplifies the template specialization generation process. by @swishdiff in This PR adds the generated kernels and simplifies the template specialization generation process. #1381
feat: High-level ak._v2.Array clean-ups. by @jpivarski in High-level ak._v2.Array clean-ups. #1392
feat: This PR adds all the remaining kernels in the studies directory by @swishdiff in This PR adds all the remaining kernels in the studies directory #1390
feat: This PR adds JAX as a new nplike by @swishdiff in This PR adds JAX as a new nplike #1399
feat: Passing behaviour in ak._v2 functions by @ioanaif in Passing behaviour in ak._v2 functions #1415
feat: Enable broadcasting of string equality. by @jpivarski in Enable broadcasting of string equality. #1427
feat: Enabled string/categorical behavior by @ioanaif in Enabled string/categorical behavior #1421
feat: Implements ak.nan_to_none and all of the ak.nan* functions to override NumPy's. by @jpivarski in Implements ak.nan_to_none and all of the ak.nan* functions to override NumPy's. #1428
feat: Pretty-printing types by @jpivarski in Pretty-printing types #1430
feat: Register both v1 and v2 Arrays in Numba entry_points. by @jpivarski in Register both v1 and v2 Arrays in Numba entry_points. #1432
feat: Add Array and Record.delitem. And fix show(type=True). by @jpivarski in Add Array and Record.__delitem__. And fix show(type=True). #1433
feat: Enable categorical behavior - testing by @ioanaif in Enable categorical behavior - testing #1434
feat: Enable mixins behavior by @ioanaif in Enable mixins behavior #1437
feat: Implementing ak._v2.to_parquet. by @jpivarski in Implementing ak._v2.to_parquet. #1440
feat: awkward to rdataframe by @ianna in awkward to rdataframe #1374
feat: Enable ak.firsts by @ioanaif in Enable ak.firsts #1443
feat: Enable ak.singletons by @ioanaif in Enable ak.singletons #1444
feat: Revamping the to_json/from_json interface. by @jpivarski in Revamping the to_json/from_json interface. #1449
feat: This PR attempts to add autodifferentiation support for Awkward Arrays using JAX pytrees. by @swishdiff in This PR attempts to add autodifferentiation support for Awkward Arrays using JAX pytrees. #1447
feat: Adding a Forth Based Avro Reader by @aryan26roy in Adding a Forth Based Avro Reader #1491
feat: Adding repr overriden behavior by @ioanaif in Adding repr overriden behavior #1487
feat: rdataframe to awkward by @ianna in rdataframe to awkward #1474
feat: add missing _like methods to TypeTracer by @agoose77 in Feat: add missing _like methods to TypeTracer #1505
feat: from rdataframe for awkward arrays by @ianna in from rdataframe for awkward arrays #1508
feat: Add typeparser to v2. by @jpivarski in Add typeparser to v2. #1513
feat: add C++ headers-only distribution configuration by @ianna in add C++ headers-only distribution configuration #1523
feat: Growable Buffer header by @ManasviGoyal in Growable Buffer header #1535
feat: parquet redux by @martindurant in parquet redux #1476
feat: Templated LayoutBuilder by @ManasviGoyal in Templated LayoutBuilder #1494
feat: raise AttributeError for public Array attributes by @agoose77 in Feat: raise AttributeError for public Array attributes #1573
feat: Public interface for layout.recursively_apply and broadcast_and_apply. by @jpivarski in Public interface for layout.recursively_apply and broadcast_and_apply. #1610
feat: prevent reducers like ak.sum on records (v2) by @ioanaif in feat: prevent reducers like ak.sum on records (v2) #1607
feat: Drop ak.behavior['.', 'Name'] = cls, which isn't working/isn't tested. by @jpivarski in feat: Drop ak.behavior['.', 'Name'] = cls, which isn't working/isn't tested. #1651
feat: retrieve multiple columns from RDataFrame in a single event loop by @ianna in feat: retrieve multiple columns from RDataFrame in a single event loop #1625

Bug-fixes

fix: Fixed ak.num with axis=0 in typetracer. by @jpivarski in Fixed ak.num with axis=0 in typetracer. #1329
fix: ak.flatten and ak.ravel should test for nplike.ndarray, not np.ndarray. by @jpivarski in ak.flatten and ak.ravel should test for nplike.ndarray, not np.ndarray. #1340
fix: Straighten out error handling via a thread-local (but otherwise global) context by @jpivarski in Straighten out error handling via a thread-local (but otherwise global) context #1327
fix: Fix PR Differing behavior depending on placement of ak.with_field #788: avoid materializing VirtualArrays in ak.with_name. by @jpivarski in Fix PR #788: avoid materializing VirtualArrays in ak.with_name. #1346
fix: fix docstring line in is_tuple (v1) by @agoose77 in Fix: fix docstring line in is_tuple (v1) #1356
fix: fixes ak.zip does not accept generator expressions for arrays #1363 by ensuring that arguments documented as 'iterable of X' aren't used in 'len(X)'. by @jpivarski in Fixes #1363 by ensuring that arguments documented as 'iterable of X' aren't used in 'len(X)'. #1364
fix: Allow NumPy arrays in CppStatements; fix row_groups in single-file from_parquet. by @jpivarski in Allow NumPy arrays in CppStatements; fix row_groups in single-file from_parquet. #1372
fix: Fixes nonlocal reducers in which the first list is empty. by @jpivarski in Fixes nonlocal reducers in which the first list is empty. #1378
fix: Fix _prettyprint after 'for i in range' changed to 'for i, val in enumerate'. by @jpivarski in Fix _prettyprint after 'for i in range' changed to 'for i, val in enumerate'. #1384
fix: bump black to 22.3.0 due to click 8.1 release by @henryiii in fix: bump black to 22.3.0 due to click 8.1 release #1385
fix: No zero-length shortcuts for ak.argsort (v1 & v2). by @jpivarski in No zero-length shortcuts for ak.argsort (v1 & v2). #1387
fix: ErrorContexts should only contain strings. by @jpivarski in ErrorContexts should only contain strings. #1393
fix: ROOT doesn't recognize for-each iterators without operator== by @jpivarski in ROOT doesn't recognize for-each iterators without operator== #1398
fix: Implement recursively_apply for Record by @agoose77 in Implement recursively_apply for Record #1401
fix: from_numpy references ListArray64 by @agoose77 in Fix: from_numpy references ListArray64 #1404
fix: Windows builds stopped working; be looser about directory name. by @jpivarski in Windows builds stopped working; be looser about directory name. #1407
fix: pypy 3.9 by @henryiii in fix: pypy 3.9 #1412
fix: replace llvmlite.ir instead of llvmlite.llvmpy.core by @Ahmad-AlSubaie in replace llvmlite.ir instead of llvmlite.llvmpy.core #1413
fix: Fix performance issue in v2 tolist. by @jpivarski in Fix performance issue in v2 tolist. #1418
fix: Fix iteration over NumpyArray type. by @jpivarski in Fix iteration over NumpyArray type. #1419
fix: Removed bytemask() in favour of mask_as_bool() by @ioanaif in Removed bytemask() in favour of mask_as_bool() #1410
fix: Fix ak._v2.to_arrow for sliced ListOffsetArray. by @jpivarski in Fix ak._v2.to_arrow for sliced ListOffsetArray. #1425
fix: ListOffsetArray._reduce_next is not implemented for 32-bit. by @jpivarski in ListOffsetArray._reduce_next is not implemented for 32-bit. #1426
fix: Fixed miscellaneous optiontype-in-Parquet bugs by @jpivarski in Fixed miscellaneous optiontype-in-Parquet bugs #1431
fix: Remove duplicated import of to/from-parquet by @douglasdavis in Remove duplicated import of to/from-parquet #1435
fix: is_unique() for IndexedArray by @ioanaif in Fix: is_unique() for IndexedArray #1429
fix: Fixes to_layout with allow_records=False and allows single-record writing to Arrow and Parquet by @jpivarski in Fixes to_layout with allow_records=False and allows single-record writing to Arrow and Parquet #1456
fix: Fix RDataFrame GetColumnNames order in test. by @jpivarski in Fix RDataFrame GetColumnNames order in test. #1457
fix: Streamline recursively_apply for small slices of big arrays. by @jpivarski in Streamline recursively_apply for small slices of big arrays. #1458
fix: getitem* functions must consistently set the slicer in handle_error. by @jpivarski in _getitem_* functions must consistently set the slicer in handle_error. #1469
fix: to_rdataframe extensive tests and bug fixes by @ianna in to_rdataframe extensive tests and bug fixes #1478
fix: Fix selecting columns from Parquet. by @jpivarski in Fix selecting columns from Parquet. #1486
fix: Fix categorical equality handling (bad copy-paste from v1). by @jpivarski in Fix categorical equality handling (bad copy-paste from v1). #1492
fix: _to_numpy method should return a numpy array by @swishdiff in _to_numpy method should return a numpy array #1496
fix: Fix/Fixed slicing shape for array of booleans by @ioanaif in Fix/Fixed slicing shape for array of booleans #1497
fix: Fixed typo in unmaskedarray.py. by @jpivarski in Fixed typo in unmaskedarray.py. #1498
fix: Fix slicing for UnmaskedArrays (which come from Arrow). by @jpivarski in Fix slicing for UnmaskedArrays (which come from Arrow). #1499
fix: Fix for issue 1406 by @ioanaif in Fix for issue 1406 #1502
fix: Update type-parser for v2 by @jpivarski in Update type-parser for v2 #1514
fix: numba pre-commit issues by @ioanaif in Fix: numba pre-commit issues #1533
fix: Initialize values behind the mask in ak.to_numpy by @ioanaif in Fix: Initialize values behind the mask in ak.to_numpy #1531
fix: Fixed RecordArray.repr (last vestige of 'override' misunderstanding). by @jpivarski in Fixed RecordArray.__repr__ (last vestige of 'override' misunderstanding). #1537
fix: Error when using ak.copy in v2 by @ioanaif in Fix: Error when using ak.copy in v2 #1532
fix: ak._v2.is_none check for axis value by @ioanaif in Fix: ak._v2.is_none check for axis value #1539
fix: Solving the endian bug on MacOS by @aryan26roy in Solving the endian bug on MacOS #1538
fix: fix ak2 convert class name msg by @Moelf in fix ak2 convert class name msg #1544
fix: GrowableBuffer.h's missing import broke localbuild.py; fixing it. by @jpivarski in GrowableBuffer.h's missing <cstring> import broke localbuild.py; fixing it. #1555
fix: Remove std::cout from ArrayBuilder code. by @jpivarski in Remove std::cout from ArrayBuilder code. #1561
fix: Missing axis_wrap_if_negative Record method in both v1 and v2 by @ioanaif in Fix: Missing axis_wrap_if_negative Record method in both v1 and v2 #1565
fix: ufuncs on records should not be allowed unless overridden by @ioanaif in Fix: ufuncs on records should not be allowed unless overridden #1559
fix: add int64_t definition for Windows by @ianna in add int64_t definition for Windows #1572
fix: allow empty RecordArrays in ak.to_layout by @agoose77 in Fix: allow empty RecordArrays in ak.to_layout #1581
fix: tolist() bug from Uproot (longlong type for Index) by @ioanaif in Fix: tolist() bug from Uproot (longlong type for Index) #1567
fix: Lengths of empty regular slices by @ioanaif in Fix: Lengths of empty regular slices #1568
fix: RegularArray: maybe_toNumpy() by @ioanaif in Fix: RegularArray: maybe_toNumpy() #1589
fix: Raises an exception on mixed Awkward/NumPy slices. by @jpivarski in Raises an exception on mixed Awkward/NumPy slices. #1598
fix: ak.concatenate (mergemany) should preserve regular-type. by @jpivarski in ak.concatenate (mergemany) should preserve regular-type. #1604
fix: ak.concatenate of identical Forms preserves the Form by @jpivarski in ak.concatenate of identical Forms preserves the Form #1605
fix: ak.concatenate should preserve regular-type for axis>0, too. by @jpivarski in ak.concatenate should preserve regular-type for axis>0, too. #1609
fix: improve ak.from_iter performance for long (axis=0) arrays by @agoose77 in fix: improve ak.from_iter performance for long (axis=0) arrays #1614
fix: generator tolayout in cling by @ianna in fix: generator tolayout in cling #1613
fix: Use dtype=np.int64, not int, for platform independence (Windows). by @jpivarski in fix: Use dtype=np.int64, not int, for platform independence (Windows). #1636
fix: support root empty field in Parquet file by @agoose77 in fix: support root empty field in Parquet file #1619
fix: empty slice lists of record arrays (RecordArray loses its field in a slice that doesn't select fields (v2) #1593) by @agoose77 in fix: empty slice lists of record arrays (#1593) #1597
fix: ak.from_iter should interpret top-level tuples as ak.Array (v1 and v2). by @jpivarski in fix: ak.from_iter should interpret top-level tuples as ak.Array (v1 and v2). #1642
fix: Ensure that ak._v2.to_json raises errors when appropriate. by @jpivarski in fix: Ensure that ak._v2.to_json raises errors when appropriate. #1649
fix: pass a copy of RecordArray's internal fields in HL API by @Saransh-cpp in fix: pass a copy of RecordArray's internal fields in HL API #1650
fix: include self._length in RegularArray.mergemany by @agoose77 in fix: include self._length in RegularArray.mergemany #1645
fix: jagged slicing for ListArray by @agoose77 in fix: jagged slicing for ListArray #1408
fix: to_list must follow getitem implementations, even in Records by @jpivarski in fix: to_list must follow __getitem__ implementations, even in Records #1652

Other

perf: ak.concatenate should flatten for first axis, single-array by @agoose77 in perf: ak.concatenate should flatten for first axis, single-array #1641
refactor: cleanup reducer by @agoose77 in Refactor: cleanup reducer #1365
refactor: split up functions by @martindurant in split up functions #1397
refactor: Flatten directory structure under src/awkward/_v2/operations. by @jpivarski in Flatten directory structure under src/awkward/_v2/operations. #1467
refactor: Rename low-level methods to match high-level function names. by @jpivarski in Rename low-level methods to match high-level function names. #1468
refactor: Rename fillna -> fill_none. by @jpivarski in Rename fillna -> fill_none. #1470
refactor: Refactoring to include index_nplike and reducers by @swishdiff in Refactoring to include index_nplike and reducers #1490
refactor: LayoutBuilder migration to v2 by @ianna in LayoutBuilder migration to v2 #1484
refactor: restructure cpp headers by @ianna in restructure cpp headers #1524
refactor: migrate ArrayBuilder to new GrowableBuffer by @ianna in migrate ArrayBuilder to new GrowableBuffer #1542
refactor: builder options and layout builder updates by @ianna in builder options and layout builder updates #1560
refactor: Renamed ak.to_pandas -> ak.to_dataframe. by @jpivarski in Renamed ak.to_pandas -> ak.to_dataframe. #1603
refactor: replace 3 ak.from_json* functions with one by @jpivarski in refactor: replace 3 ak.from_json* functions with one #1617
refactor: from_rdataframe to use LayoutBuilder by @ianna in refactor: from_rdataframe to use LayoutBuilder #1620
docs: add Ahmad-AlSubaie as a contributor for code by @allcontributors in docs: add Ahmad-AlSubaie as a contributor for code #1416
docs: Update AwkwardForth documentation and move it from the wiki to the standard docs. by @jpivarski in Update AwkwardForth documentation and move it from the wiki to the standard docs. #1422
docs: update README: from awkward-1.0 to awkward by @ianna in update README: from awkward-1.0 to awkward #1475
docs: Add the beginning of an example of templated LayoutBuilder. by @jpivarski in Add the beginning of an example of templated LayoutBuilder. #1485
docs: add ManasviGoyal as a contributor for code by @allcontributors in docs: add ManasviGoyal as a contributor for code #1540
docs: add aryan26roy as a contributor for code by @allcontributors in docs: add aryan26roy as a contributor for code #1541
docs: Doxygen Documentation of GrowableBuffer and LayoutBuilder. by @ManasviGoyal in Doxygen Documentation of GrowableBuffer and LayoutBuilder. #1579
docs: Documented ak.from_json and completed it. by @jpivarski in docs: Documented ak.from_json and completed it. #1621
docs: add Saransh-cpp as a contributor for code by @allcontributors in docs: add Saransh-cpp as a contributor for code #1653
ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in [pre-commit.ci] pre-commit autoupdate #1370
ci: Protect test 1300 from ROOT without C++17 (or, at least, without std::optional). by @jpivarski in Protect test 1300 from ROOT without C++17 (or, at least, without std::optional). #1383
ci: Build(deps): bump pypa/cibuildwheel from 2.3.1 to 2.4.0 by @dependabot in Build(deps): bump pypa/cibuildwheel from 2.3.1 to 2.4.0 #1394
ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in [pre-commit.ci] pre-commit autoupdate #1395
ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in [pre-commit.ci] pre-commit autoupdate #1409
ci: Pass on skipped v2 tests by @ioanaif in Pass on skipped v2 tests #1445
ci: Build(deps): bump pypa/cibuildwheel from 2.4.0 to 2.5.0 by @dependabot in Build(deps): bump pypa/cibuildwheel from 2.4.0 to 2.5.0 #1455
ci: awkward rdataframe source tests by @ianna in awkward rdataframe source tests #1446
ci: Ignore a NumPy 1.22 warning in Numba and fix the flake8-print T001 --> T201 change. by @jpivarski in Ignore a NumPy 1.22 warning in Numba and fix the flake8-print T001 --> T201 change. #1464
ci: Build(deps): bump docker/setup-qemu-action from 1.2.0 to 2.0.0 by @dependabot in Build(deps): bump docker/setup-qemu-action from 1.2.0 to 2.0.0 #1462
ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in [pre-commit.ci] pre-commit autoupdate #1465
ci: Build(deps): bump pypa/cibuildwheel from 2.5.0 to 2.6.1 by @dependabot in Build(deps): bump pypa/cibuildwheel from 2.5.0 to 2.6.1 #1493
ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in [pre-commit.ci] pre-commit autoupdate #1503
ci: Build(deps): bump pypa/cibuildwheel from 2.6.1 to 2.7.0 by @dependabot in Build(deps): bump pypa/cibuildwheel from 2.6.1 to 2.7.0 #1510
ci: Swap 'merged dtype same as NumPy' test of v1 for test of v2. by @jpivarski in Swap 'merged dtype same as NumPy' test of v1 for test of v2. #1515
ci: Build(deps): bump actions/upload-artifact from 2 to 3 by @dependabot in Build(deps): bump actions/upload-artifact from 2 to 3 #1519
ci: Build(deps): bump actions/checkout from 2 to 3 by @dependabot in Build(deps): bump actions/checkout from 2 to 3 #1520
ci: Build(deps): bump actions/download-artifact from 2 to 3 by @dependabot in Build(deps): bump actions/download-artifact from 2 to 3 #1521
ci: Build(deps): bump pypa/cibuildwheel from 2.7.0 to 2.8.0 by @dependabot in Build(deps): bump pypa/cibuildwheel from 2.7.0 to 2.8.0 #1536
ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in [pre-commit.ci] pre-commit autoupdate #1527
ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in [pre-commit.ci] pre-commit autoupdate #1543
ci: Build(deps): bump pypa/cibuildwheel from 2.8.0 to 2.8.1 by @dependabot in Build(deps): bump pypa/cibuildwheel from 2.8.0 to 2.8.1 #1556
ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in [pre-commit.ci] pre-commit autoupdate #1566
ci: [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in [pre-commit.ci] pre-commit autoupdate #1576
ci: Build(deps): bump pypa/gh-action-pypi-publish from 1.5.0 to 1.5.1 by @dependabot in Build(deps): bump pypa/gh-action-pypi-publish from 1.5.0 to 1.5.1 #1564
ci: use GHA instead of Azure 🔨 by @agoose77 in ci: use GHA instead of Azure 🔨 #1550
ci: Upgrade Linux 18.04 -> 20.04 and DOS line endings. by @jpivarski in ci: Upgrade Linux 18.04 -> 20.04 and DOS line endings. #1616
ci: lint PR titles according to conventional commits by @agoose77 in ci: lint PR titles according to conventional commits #1615
ci: aarch64 should also use cibuildwheel v2.9.0 by @jpivarski in ci: aarch64 should also use cibuildwheel v2.9.0 #1627
ci: run semantic-pr-title in concurrency group by @agoose77 in ci: run semantic-pr-title in concurrency group #1632
ci: add Linux ROOT build to run ROOT-based tests by @ianna in ci: add Linux ROOT build to run ROOT-based tests #1629
package: fix MANIFEST to include cpp-headers by @agoose77 in Package: fix MANIFEST to include cpp-headers #1516
chore: remove unneeded lines by @henryiii in chore: remove unneeded lines #1518
chore: include Python 3.11 by @henryiii in chore: include Python 3.11 #1602
chore: update pre-commit hooks by @pre-commit-ci in chore: update pre-commit hooks #1618

New Contributors

@Ahmad-AlSubaie made their first contribution in replace llvmlite.ir instead of llvmlite.llvmpy.core #1413
@martindurant made their first contribution in split up functions #1397
@aryan26roy made their first contribution in Adding a Forth Based Avro Reader #1491
@ManasviGoyal made their first contribution in Growable Buffer header #1535
@Moelf made their first contribution in fix ak2 convert class name msg #1544

Full Changelog: 1.8.0...v1.9.0

This discussion was created from the release Version 1.9.0.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Version 1.9.0 #1664

Uh oh!

{{title}}

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

Version 1.9.0 #1664

Uh oh!

jpivarski Sep 2, 2022 Maintainer

Features

Bug-fixes

Other

New Contributors

Replies: 0 comments

jpivarski
Sep 2, 2022
Maintainer