Merge pull request #66 from swansonk14/Py310Unions

Union types in Python 3.10

Author: swansonk14

.github/workflows/code-coverage.yml

@@ -6,13 +6,13 @@ jobs:
   run:
     runs-on: ubuntu-latest
     env:
-      PYTHON: '3.9'
+      PYTHON: '3.10'
     steps:
     - uses: actions/checkout@main
     - name: Setup Python
       uses: actions/setup-python@main
       with:
-        python-version: 3.9
+        python-version: '3.10'
     - name: Generate coverage report
       run: |
         git config --global user.email "you@example.com"

.github/workflows/tests.yml

@@ -16,7 +16,7 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
-        python-version: [3.6, 3.7, 3.8, 3.9]
+        python-version: ['3.6', '3.7', '3.8', '3.9', '3.10']
 
     steps:
     - uses: actions/checkout@v2

README.md

@@ -41,25 +41,38 @@ pip install -e .
 ## Table of Contents
 
 * [Installation](#installation)
+* [Table of Contents](#table-of-contents)
 * [Tap is Python-native](#tap-is-python-native)
 * [Tap features](#tap-features)
-    + [Arguments](#arguments)
-    + [Help string](#help-string)
-    + [Flexibility of `configure`](#flexibility-of-configure)
-    + [Types](#types)
-    + [Argument processing with `process_args`](#argument-processing-with-process_args)
-    + [Processing known args](#processing-known-args)
-    + [Subclassing](#subclassing)
-    + [Printing](#printing)
-    + [Reproducibility](#reproducibility)
-      - [Reproducibility info](#reproducibility-info)
-    + [Saving and loading arguments](#saving-and-loading-arguments)
-      - [Save](#save)
-      - [Load](#load)
-      - [Load from dict](#load-from-dict)
-    + [Loading from configuration files](#loading-from-configuration-files)
+  + [Arguments](#arguments)
+  + [Help string](#help-string)
+  + [Flexibility of `configure`](#flexibility-of--configure-)
+    - [Adding special argument behavior](#adding-special-argument-behavior)
+    - [Adding subparsers](#adding-subparsers)
+  + [Types](#types)
+    - [`str`, `int`, and `float`](#-str----int---and--float-)
+    - [`bool`](#-bool-)
+    - [`Optional`](#-optional-)
+    - [`List`](#-list-)
+    - [`Set`](#-set-)
+    - [`Tuple`](#-tuple-)
+    - [`Literal`](#-literal-)
+    - [`Union`](#-union-)
+    - [Complex Types](#complex-types)
+  + [Argument processing with `process_args`](#argument-processing-with--process-args-)
+  + [Processing known args](#processing-known-args)
+  + [Subclassing](#subclassing)
+  + [Printing](#printing)
+  + [Reproducibility](#reproducibility)
+    - [Reproducibility info](#reproducibility-info)
+  + [Saving and loading arguments](#saving-and-loading-arguments)
+    - [Save](#save)
+    - [Load](#load)
+    - [Load from dict](#load-from-dict)
+  + [Loading from configuration files](#loading-from-configuration-files)
 
 ## Tap is Python-native
+
 To see this, let's look at an example:
 
 ```python
@@ -202,7 +215,7 @@ class Args(Tap):
 
 ### Types
 
-Tap automatically handles all of the following types:
+Tap automatically handles all the following types:
 
 ```python
 str, int, float, bool
@@ -215,8 +228,10 @@ Literal
 
 If you're using Python 3.9+, then you can replace `List` with `list`, `Set` with `set`, and `Tuple` with `tuple`.
 
+Tap also supports `Union`, but this requires additional specification (see [Union](#-union-) section below).
+
 Additionally, any type that can be instantiated with a string argument can be used. For example, in
-```
+```python
 from pathlib import Path
 from tap import Tap
 
@@ -257,21 +272,62 @@ Tuples can be used to specify a fixed number of arguments with specified types u
 
 Literal is analagous to argparse's [choices](https://docs.python.org/3/library/argparse.html#choices), which specifies the values that an argument can take. For example, if arg can only be one of 'H', 1, False, or 1.0078 then you would specify that `arg: Literal['H', 1, False, 1.0078]`. For instance, `--arg False` assigns arg to False and `--arg True` throws error. The `Literal` type was introduced in Python 3.8 ([PEP 586](https://www.python.org/dev/peps/pep-0586/)) and can be imported with `from typing_extensions import Literal`.
 
-#### Complex types
+#### `Union`
 
-More complex types _must_ be specified with the `type` keyword argument in `add_argument`, as in the example below.
+Union types must include the `type` keyword argument in `add_argument` in order to specify which type to use, as in the example below.
 
 ```python
-def to_number(string: str):
+def to_number(string: str) -> Union[float, int]:
     return float(string) if '.' in string else int(string)
 
 class MyTap(Tap):
-    number: Union[int, float]
+    number: Union[float, int]
 
     def configure(self):
         self.add_argument('--number', type=to_number)
 ```
 
+In Python 3.10+, `Union[Type1, Type2, etc.]` can be replaced with `Type1 | Type2 | etc.`, but the `type` keyword argument must still be provided in `add_argument`.
+
+#### Complex Types
+
+Tap can also support more complex types than the ones specified above. If the desired type is constructed with a single string as input, then the type can be specified directly without additional modifications. For example,
+
+```python
+class Person:
+    def __init__(self, name: str) -> None:
+        self.name = name
+
+class Args(Tap):
+    person: Person
+
+args = Args().parse_args('--person Tapper'.split())
+print(args.person.name)  # Tapper
+```
+
+If the desired type has a more complex constructor, then the `type` keyword argument must be provided in `add_argument`. For example,
+
+```python
+class AgedPerson:
+    def __init__(self, name: str, age: int) -> None:
+        self.name = name
+        self.age = age
+
+def to_aged_person(string: str) -> AgedPerson:
+    name, age = string.split(',')
+    return AgedPerson(name=name, age=int(age))
+
+class Args(Tap):
+    aged_person: AgedPerson
+
+    def configure(self) -> None:
+        self.add_argument('--aged_person', type=to_aged_person)
+
+args = Args().parse_args('--aged_person Tapper,27'.split())
+print(f'{args.aged_person.name} is {args.aged_person.age}')  # Tapper is 27
+```
+
+
 ### Argument processing with `process_args`
 
 With complex argument parsing, arguments often end up having interdependencies. This means that it may be necessary to disallow certain combinations of arguments or to modify some arguments based on other arguments.

tap/__init__.py

@@ -1,4 +1,5 @@
+from argparse import ArgumentError, ArgumentTypeError
 from tap._version import __version__
 from tap.tap import Tap
 
-__all__ = ['Tap', '__version__']
+__all__ = ['ArgumentError', 'ArgumentTypeError', 'Tap', '__version__']

tap/tap.py

@@ -1,4 +1,4 @@
-from argparse import ArgumentParser
+from argparse import ArgumentParser, ArgumentTypeError
 from collections import OrderedDict
 from copy import deepcopy
 from functools import partial
@@ -10,11 +10,12 @@
 import time
 from types import MethodType
 from typing import Any, Callable, Dict, List, Optional, Sequence, Set, Tuple, TypeVar, Union, get_type_hints
-from typing_inspect import is_literal_type, get_args
+from typing_inspect import is_literal_type
 from warnings import warn
 
 from tap.utils import (
     get_class_variables,
+    get_args,
     get_argument_name,
     get_dest,
     get_origin,
@@ -30,11 +31,15 @@
     enforce_reproducibility,
 )
 
+if sys.version_info >= (3, 10):
+    from types import UnionType
+
 
 # Constants
 EMPTY_TYPE = get_args(List)[0] if len(get_args(List)) > 0 else tuple()
 BOXED_COLLECTION_TYPES = {List, list, Set, set, Tuple, tuple}
-OPTIONAL_TYPES = {Optional, Union}
+UNION_TYPES = {Union} | ({UnionType} if sys.version_info >= (3, 10) else set())
+OPTIONAL_TYPES = {Optional} | UNION_TYPES
 BOXED_TYPES = BOXED_COLLECTION_TYPES | OPTIONAL_TYPES
 
 
@@ -169,13 +174,30 @@ def _add_argument(self, *name_or_flags, **kwargs) -> None:
 
             # If type is not explicitly provided, set it if it's one of our supported default types
             if 'type' not in kwargs:
-
-                # Unbox Optional[type] and set var_type = type
+                # Unbox Union[type] (Optional[type]) and set var_type = type
                 if get_origin(var_type) in OPTIONAL_TYPES:
                     var_args = get_args(var_type)
 
+                    # If type is Union or Optional without inner types, set type to equivalent of Optional[str]
+                    if len(var_args) == 0:
+                        var_args = (str, type(None))
+
+                    # Raise error if type function is not explicitly provided for Union types (not including Optionals)
+                    if get_origin(var_type) in UNION_TYPES and not (len(var_args) == 2 and var_args[1] == type(None)):
+                        raise ArgumentTypeError(
+                            'For Union types, you must include an explicit type function in the configure method. '
+                            'For example,\n\n'
+                            'def to_number(string: str) -> Union[float, int]:\n'
+                            '    return float(string) if \'.\' in string else int(string)\n\n'
+                            'class Args(Tap):\n'
+                            '    arg: Union[float, int]\n'
+                            '\n'
+                            '    def configure(self) -> None:\n'
+                            '        self.add_argument(\'--arg\', type=to_number)'
+                        )
+
                     if len(var_args) > 0:
-                        var_type = get_args(var_type)[0]
+                        var_type = var_args[0]
 
                         # If var_type is tuple as in Python 3.6, change to a typing type
                         # (e.g., (typing.List, <class 'bool'>) ==> typing.List[bool])
@@ -187,21 +209,23 @@ def _add_argument(self, *name_or_flags, **kwargs) -> None:
                 # First check whether it is a literal type or a boxed literal type
                 if is_literal_type(var_type):
                     var_type, kwargs['choices'] = get_literals(var_type, variable)
+
                 elif (get_origin(var_type) in (List, list, Set, set)
                       and len(get_args(var_type)) > 0
                       and is_literal_type(get_args(var_type)[0])):
                     var_type, kwargs['choices'] = get_literals(get_args(var_type)[0], variable)
                     if kwargs.get('action') not in {'append', 'append_const'}:
                         kwargs['nargs'] = kwargs.get('nargs', '*')
+
                 # Handle Tuple type (with type args) by extracting types of Tuple elements and enforcing them
                 elif get_origin(var_type) in (Tuple, tuple) and len(get_args(var_type)) > 0:
                     loop = False
                     types = get_args(var_type)
 
                     # Don't allow Tuple[()]
                     if len(types) == 1 and types[0] == tuple():
-                        raise ValueError('Empty Tuples (i.e. Tuple[()]) are not a valid Tap type '
-                                         'because they have no arguments.')
+                        raise ArgumentTypeError('Empty Tuples (i.e. Tuple[()]) are not a valid Tap type '
+                                                'because they have no arguments.')
 
                     # Handle Tuple[type, ...]
                     if len(types) == 2 and types[1] == Ellipsis:
@@ -237,7 +261,7 @@ def _add_argument(self, *name_or_flags, **kwargs) -> None:
                         kwargs['type'] = boolean_type
                         kwargs['choices'] = [True, False]  # this makes the help message more helpful
                     else:
-                        action_cond = "true" if kwargs.get("required", False) or not kwargs["default"] else "false"
+                        action_cond = 'true' if kwargs.get('required', False) or not kwargs['default'] else 'false'
                         kwargs['action'] = kwargs.get('action', f'store_{action_cond}')
                 elif kwargs.get('action') not in {'count', 'append_const'}:
                     kwargs['type'] = var_type

tap/utils.py

@@ -24,8 +24,10 @@
     Union,
 )
 from typing_extensions import Literal
-from typing_inspect import get_args, get_origin as typing_inspect_get_origin
+from typing_inspect import get_args as typing_inspect_get_args, get_origin as typing_inspect_get_origin
 
+if sys.version_info >= (3, 10):
+    from types import UnionType
 
 NO_CHANGES_STATUS = """nothing to commit, working tree clean"""
 PRIMITIVES = (str, int, float, bool)
@@ -255,7 +257,7 @@ def get_literals(literal: Literal, variable: str) -> Tuple[Callable[[str], Any],
     literals = list(get_args(literal))
 
     if not all(isinstance(literal, PRIMITIVES) for literal in literals):
-        raise ValueError(
+        raise ArgumentTypeError(
             f'The type for variable "{variable}" contains a literal'
             f'of a non-primitive type e.g. (str, int, float, bool).\n'
             f'Currently only primitive-typed literals are supported.'
@@ -264,7 +266,7 @@ def get_literals(literal: Literal, variable: str) -> Tuple[Callable[[str], Any],
     str_to_literal = {str(literal): literal for literal in literals}
 
     if len(literals) != len(str_to_literal):
-        raise ValueError('All literals must have unique string representations')
+        raise ArgumentTypeError('All literals must have unique string representations')
 
     def var_type(arg: str) -> Any:
         return str_to_literal.get(arg, arg)
@@ -403,7 +405,7 @@ def as_python_object(dct: Any) -> Any:
             return UnpicklableObject()
 
         else:
-            raise ValueError(f'Special type "{_type}" not supported for JSON loading.')
+            raise ArgumentTypeError(f'Special type "{_type}" not supported for JSON loading.')
 
     return dct
 
@@ -471,7 +473,7 @@ def enforce_reproducibility(saved_reproducibility_data: Optional[Dict[str, str]]
                          f'in current args.')
 
 
-# TODO: remove this once typing_inspect.get_origin is fixed for Python 3.8 and 3.9
+# TODO: remove this once typing_inspect.get_origin is fixed for Python 3.8, 3.9, and 3.10
 # https://github.com/ilevkivskyi/typing_inspect/issues/64
 # https://github.com/ilevkivskyi/typing_inspect/issues/65
 def get_origin(tp: Any) -> Any:
@@ -481,4 +483,16 @@ def get_origin(tp: Any) -> Any:
     if origin is None:
         origin = tp
 
+    if sys.version_info >= (3, 10) and isinstance(origin, UnionType):
+        origin = UnionType
+
     return origin
+
+
+# TODO: remove this once typing_insepct.get_args is fixed for Python 3.10 union types
+def get_args(tp: Any) -> Tuple[type, ...]:
+    """Same as typing_inspect.get_args but fixes Python 3.10 union types."""
+    if sys.version_info >= (3, 10) and isinstance(tp, UnionType):
+        return tp.__args__
+
+    return typing_inspect_get_args(tp)