optree v0.10.0

optree v0.10.0

Added

• Add tree_ravel function for JAX/NumPy/PyTorch array/tensor tree manipulation by @XuehaiPan in #100.
• Expose node kind enum for PyTreeSpec by @XuehaiPan in #98.
• Expose function tree_flatten_one_level by @XuehaiPan in #101.
• Add tree broadcast functions broadcast_common, tree_broadcast_common, tree_broadcast_map, and tree_broadcast_map_with_path by @XuehaiPan in #87.
• Add function tree_is_leaf and add is_leaf argument to function all_leaves by @XuehaiPan in #93.
• Add methods PyTreeSpec.entry and PyTreeSpec.child by @XuehaiPan in #88.
• Add Python 3.12 support by @XuehaiPan in #90.
• Allow passing third-party dependency version from environment variable by @XuehaiPan in #80.

Changed

• Set recursion limit to 2000 for all platforms by @XuehaiPan in #97.
• Make PyTreeSpec.is_prefix to be consistent with PyTreeSpec.flatten_up_to by @XuehaiPan in #94.
• Decrease the MAX_RECURSION_DEPTH to 2000 on Windows by @XuehaiPan in #85.
• Bump abseil-cpp version to 20230802.1 by @XuehaiPan in #80.

Fixed

• Memorize ongoing repr / hash calls to resolve infinite recursion under self-referential case by @XuehaiPan and @JieRen98 in #82.

Removed

• Remove dependence on abseil-cpp by @XuehaiPan in #85.

Full Changelog: v0.9.2...v0.10.0

optree v0.9.2

Patch Release [0.9.2] - 2023-09-18

Changed

• Bump pybind11 version to 2.11.1 and add initial Python 3.12 support by @XuehaiPan in #78.
• Bump abseil-cpp version to 20230802.0 by @XuehaiPan in #79.

Fixed

• Fix empty paths when flatten with custom is_leaf function by @XuehaiPan in #76.

Full Changelog: v0.9.1...v0.9.2

optree v0.9.1

Patch Release [0.9.1] - 2023-05-23

Changed

• Use py::type::handle_of(obj) rather than deprecated obj.get_type() by @XuehaiPan in #49.
• Bump abseil-cpp version to 20230125.3 by @XuehaiPan in #57.

Fixed

• Add @runtime_checkable decorator for CustomTreeNode protocol class by @XuehaiPan in #56.

Full Changelog: v0.9.0...v0.9.1

optree v0.9.0

What's New

Now optree preserves dict / defaultdict key order in the output of tree_unflatten, tree_map, and tree_map_with_path.

This behavior needs extra operations during flattening/unflattening. There would be performance regression for dict and defaultdict. No impact is added for other collection types, such as OrderedDict, list, tuple.

optree v0.8.0: the output dict from tree_unflatten is stored in sorted order.

tree_flatten and tree_unflatten will lose the information above the insertion order of the input dict. Map over a dict with the identity function will change the key order.

>>> optree.tree_map(lambda x: x, {'b': 2, 'a': 1})
{'a': 1, 'b': 2}

optree v0.9.0: the output dict from tree_unflatten will have the consistent key order with the input dict.

>>> optree.tree_map(lambda x: x, {'b': 2, 'a': 1})
{'b': 2, 'a': 1}

As the key order is preserved, it's safe to use tree_map to process "unordered dict" (builtins.dict) (e.g., **kwargs in functions). In this case, users no longer need to convert dict to OrderedDict manually.

def func(*args, **kwargs):
    args, kwargs = optree.tree_map(do_something, (args, kwargs))
    # optree v0.8.0: args, kwargs = optree.tree_map(do_something, (args, OrderedDict(kwargs)))
    ...

Note that tree_map still maps the leaves in sorted key order (the same order as tree_flatten and tree_leaves).

>>> leaves = [] 
...
... def add_leaves(x):
...     leaves.append(x)
...     return x
...
>>> ptree.tree_map(add_leaves, {'b': 2, 'a': 1})
{'b': 2, 'a': 1}
>>> leaves
[1, 2]

Breaking Changes

Revert "Change keyword argument initial to initializer for tree_reduce to align with functools.reduce". The previous change is reverted due to a documentation issue for Python (see also python/cpython#102757 and python/cpython#102759). The argument name is changed back to initial in tree_reduce. Users are recommended to use positional arguments.

Full Changelog [0.9.0] - 2023-03-23

Added

• Preserve dict key order in the output of tree_unflatten, tree_map, and tree_map_with_path by @XuehaiPan in #46.

Changed

• Change keyword argument initializer back to initial for tree_reduce to align with functools.reduce C implementation by @XuehaiPan in #47.

Full Changelog: v0.8.0...v0.9.0

optree v0.8.0

What's New

1. Add new and refactor utility functions for namedtuple and PyStructSequence types.

   • is_namedtuple
   • is_namedtuple_class
   • is_structseq
   • is_structseq_class
   • namedtuple_fields (new)
   • structseq_fields

2. Add more methods and properties for PyTreeSpec class.

   • PyTreeSpec.is_prefix (treespec_is_prefix and __le__ / __lt__)
   • PyTreeSpec.is_suffix (treespec_is_suffix and __ge__ / __gt__)
   • PyTreeSpec.paths (treespec_paths)
   • PyTreeSpec.entries (treespec_entries)

3. Add tree reduce functions.

   • tree_sum
   • tree_max
   • tree_min

4. Miscellaneous changes.

   • Rephrase error messages.

     >>> optree.tree_map(lambda x, y: x + y, {'a': 1, 'b': 2}, {'b': 3, 'c': 4})
     Traceback (most recent call last):
         ...
     ValueError: dictionary key mismatch; expected key(s): ['a', 'b'], got key(s): ['b', 'c'], missing key(s): ['a'], extra key(s): ['c']; dict: {'b': 3, 'c': 4}.

   • Add function tree_broadcast_prefix.

   • Linter integration for flake8 plugins and ruff.

Breaking Changes

1. PyTreeSpec.flatten_up_to now allows ordered (OrderedDict) and unordered (dict and defaultdict) dictionaries mutually exchangeable.

   This behavior will affect all functions based on flatten_up_to, such as tree_map, tree_map_with_path, tree_broadcast_prefix.
   Calling multi-treemap with mixed inputs of ordered and unordered dictionaries will no longer raise ValueError.

   optree v0.7.0: raise ValueError if the dictionary type mismatch

   >>> from collections import *
   >>> import optree
   >>> d = {'a': 1, 'b': 2, 'c': 3}
   >>> od = OrderedDict([('b', 5), ('c', 6), ('a', 4)])
   >>> dd = defaultdict(int, {'c': 9, 'a': 7, 'b': 8})
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, d, od, dd)
   Traceback (most recent call last):
       ...
   ValueError: Expected dict, got OrderedDict([('b', 5), ('c', 6), ('a', 4)]).
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, od, dd, d)
   Traceback (most recent call last):
       ...
   ValueError: Expected collections.OrderedDict, got defaultdict(<class 'int'>, {'c': 9, 'a': 7, 'b': 8}).
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, od, dd, d)
   Traceback (most recent call last):
       ...
   ValueError: Expected collections.defaultdict, got {'a': 1, 'b': 2, 'c': 3}.

   optree v0.8.0: allow mixed input of dict, OrderedDict, and defaultdict.
   The result pytree type and key ordering are determined by the first input pytree.

   >>> from collections import *
   >>> import optree
   >>> d = {'a': 1, 'b': 2, 'c': 3}
   >>> od = OrderedDict([('b', 5), ('c', 6), ('a', 4)])
   >>> dd = defaultdict(int, {'c': 9, 'a': 7, 'b': 8})
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, d, od, dd)
   {'a': 147, 'b': 258, 'c': 369}
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, od, dd, d)
   OrderedDict([('b', 582), ('c', 693), ('a', 471)])
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, dd, d, od)
   defaultdict(<class 'int'>, {'a': 714, 'b': 825, 'c': 936})

2. Use more appropriate exception types.

   • structseq_fields now raises TypeError rather than ValueError when got non-PyStructSequence types

     optree v0.7.0: raise ValueError when got non-PyStructSequence types

     >>> from collections import *
     >>> import optree
     >>> optree.structseq_fields((1, 2))
     Traceback (most recent call last):
         ...
     ValueError: Expected StructSequence, got (1, 2).
     >>> mytuple = namedtuple('mytuple', ['a', 'b'])
     >>> optree.structseq_fields(mytuple(1, 2))
     Traceback (most recent call last):
         ...
     ValueError: Expected StructSequence, got mytuple(a=1, b=2).

     optree v0.8.0: raise TypeError when got non-PyStructSequence types

     >>> from collections import *
     >>> import optree
     >>> optree.structseq_fields((1, 2))
     Traceback (most recent call last):
         ...
     TypeError: Expected an instance of PyStructSequence type, got (1, 2).
     >>> mytuple = namedtuple('mytuple', ['a', 'b'])
     >>> optree.structseq_fields(mytuple(1, 2))
     Traceback (most recent call last):
         ...
     TypeError: Expected an instance of PyStructSequence type, got mytuple(a=1, b=2).

   • optree._C.InternalError now inherits from SystemError rather than RuntimeError.

     optree v0.7.0: need to use RuntimeError to catch InternalError

     >>> import optree
     >>> optree._C.InternalError.mro()
     [<class 'optree._C.InternalError'>, <class 'RuntimeError'>, <class 'Exception'>, <class 'BaseException'>, <class 'object'>]

     optree v0.8.0: need to use SystemError to catch InternalError

     >>> import optree
     >>> optree._C.InternalError.mro()
     [<class 'optree._C.InternalError'>, <class 'SystemError'>, <class 'Exception'>, <class 'BaseException'>, <class 'object'>]

3. Change keyword argument initial to initializer for tree_reduce to align with functools.reduce.

Full Changelog [0.8.0] - 2023-03-14

Added

• Add methods PyTreeSpec.paths and PyTreeSpec.entries by @XuehaiPan in #43.
• Allow tree-map with mixed inputs of ordered and unordered dictionaries by @XuehaiPan in #42.
• Add more utility functions for namedtuple and PyStructSequence type by @XuehaiPan in #41.
• Add methods PyTreeSpec.is_prefix and PyTreeSpec.is_suffix and function tree_broadcast_prefix by @XuehaiPan in #40.
• Add tree reduce functions tree_sum, tree_max, and tree_min by @XuehaiPan in #39.
• Test dict key equality with PyDict_Contains ($O (n)$) rather than sorting ($O (n \log n)$) by @XuehaiPan in #37.
• Make error message more clear when value mismatch by @XuehaiPan in #36.
• Add ruff and flake8 plugins integration by @XuehaiPan in #33.

Changed

• Allow tree-map with mixed inputs of ordered and unordered dictionaries by @XuehaiPan in #42.
• Use more appropriate exception handling (e.g., change ValueError to TypeError in structseq_fields) by @XuehaiPan in #41.
• Inherit optree._C.InternalError from SystemError rather than RuntimeError by @XuehaiPan in #41.
• Change keyword argument initial to initializer for tree_reduce to align with functools.reduce by @XuehaiPan in #39.

Full Changelog: v0.7.0...v0.8.0

optree v0.7.0

[0.7.0] - 2023-02-07

Added

• Add PyStructSequence types as internal node types by @XuehaiPan in #30.

Changed

• Add PyStructSequence types as internal node types by @XuehaiPan in #30.
• Use postponed evaluation of annotations by @XuehaiPan in #28.

Full Changelog: v0.6.0...v0.7.0