v1 function-oriented library

[alkasm]

This PR adds a function-oriented interface around the new SDK for simple and short high-level scripting. This is intended to replace the prior v0 client library.

Migration guide

The initial v0 client library and the new v1 are architected differently, so there isn't a direct 1-to-1 translation. However, the v1 has the same capabilities and more. Some examples are given below for transitioning to the new v1. Please contact your friendly neighborhood Nomineer if you'd like some help translating to the new client library!

Everything you need is now under the main nominal namespace: import nominal as nm

Set your token

v0

nm.cloud.set_token("eyJ...")

v1
Instead of setting this token in Python, you set it at the command-line:

python3 -m nominal auth set-token

Create and upload a dataset

The v0 library tried to guess the timestamp column, but you could also specify it manually. Different versions looked like:

v0

dataset = Ingest().read("data.csv")
dataset = Ingest.set_ts_index(dataset)
dataset.upload()

# or

dataset = Ingest().read("data.csv", ts_col="timestamp")
dataset.upload()

# or

df = pl.read_csv("data.csv")
df = Ingest.set_ts_index(df, ts_col="timestamp")
dataset = Dataset(df, "data.csv", relative=True, relative_units="SECONDS")
dataset.upload()

The v1 library requires you to specify the timestamp column and type:

v1

dataset = nm.upload_csv(
    "data.csv",
    "Dataset Name",
    timestamp_column="timestamp",
    timestamp_type="iso_8601",  # or "epoch_seconds", "relative_microseconds", etc
)

There are also nm.upload_polars() and nm.upload_pandas() functions.

Get a dataset

v0
Not available

v1

dataset = nm.get_dataset("ri.catalog....")

Get a run

The initial client library had slightly different functionality depending on how you created the Run object. You could get a run by its rid, if you didn't specify files to upload:

v0

run = Run(rid="ri.scout.run...")

v1

run = nm.get_run("ri.scout.run...")

Create a run

v0

run = Run(path="data.csv")  # from a single CSV file
run = Run(paths=["data1.csv", "data2.csv"])  # from multiple CSV files
run = Run(datasets=[dataset_a, dataset_b])  # from multiple datasets that were already uploaded

v1

# create a run without datasets (optionally add them later)

run = nm.create_run("My Run", start="2024-09-05T12:35:00Z", end="2024-09-05T13:15:00Z")
run.add_dataset("test-stand-a", dataset_a)
run.add_dataset("test-stand-b", dataset_b)

# or, create a run and a dataset simultaneously

run = nm.create_run_csv(
    "data.csv",
    "My Run",
    timestamp_column="timestamp",
    timestamp_type="iso_8601",
)

The v1 client library does not support creating a run with multiple CSV files simultaneously. Instead, use nm.create_run() with run.add_dataset() to add multiple datasets.

Set the base URL

Generally, users aren't expected to use this; it's more for Nomineers for local development or staging environments.

v0

nm.cloud.set_base_url("PROD")

v1

nm.set_base_url("https://api.gov.nominal.io/api")

[alxhill]

pushing this out as a checkpoint, will switch to comments vs a review given size

some of these comments might be answered later in the code, so feel free to ignore while I continue reviewing if there are obvious oversights

[alkasm]

Keeping the migration guide as the top-level comment. Here's the rest of the PR info:

• top-level functions for most common ops (create, get, list, update on datasets, runs, attachments)
• e2e test every top level function
• use a custom type for (file type, mime type) pairs in SDK
• consistent ordering of labels, props in SDK
• remove ability to create runs with datasets simultaneously in SDK and top-level
• change run.add_datasets() to run.add_dataset()
• add get_attachments() and list_attachments() to SDK
• bump minimum python version to 3.9 (some dependencies, like pandas, don't support 3.8 anymore)
• overhaul README
• write/read through top-level docstrings
• map v0 functions to v1 functions
• poll for completion on all dataset uploads on top level functions
• use "name" over "title" for everything
• remove data (should add back later with a focus on timeseries data)
• add function for going straight from csv -> (dataset and run)

Testing

% just validate-e2e eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2F1dGgubm9taW5hbC50ZXN0Iiwic3ViIjoicmkuYXV0aG4uZGV2LnVzZXIuOGRlN2Q5M2MtNzU0Ny00M2Q5LWIyYjgtY2IzMGQwYzhmYTBhIiwibm9taW5hbCI6eyJ1c2VyX3V1aWQiOiI4ZGU3ZDkzYy03NTQ3LTQzZDktYjJiOC1jYjMwZDBjOGZhMGEiLCJvcmdhbml6YXRpb25fdXVpZCI6Ijc5YzJjZGQxLTRjY2QtNDI4OC1iMDZiLTI0ZTkwMmE2YjNhZiJ9LCJleHAiOjE3MjYwMzAyNzIsImF1ZCI6Imh0dHBzOi8vYXBpLm5vbWluYWwudGVzdCJ9.vQmczbf5cC4YJpPKLOMoXOwhG8CyHCN--9E3YczImSdFl3B0pULav0u6aN73PvqWVAgDYGw-Tb4XUlLH8DLkLicXVop-UN80SvseyllnMN8WdPRT6NmSVrEjKaGoTCpyRrAUCKowgEUn3owzoJACJM1TY0PhpUj8hJ5F7psKJ9DzjnqqUF2wZ_aXQ_ACu7xcOm1nC1wS1aSDnRuc8BbP38S9StfTlDwNStelzVP_QHhwKPuq0mnUY9NuziNKOXhDjHK3OOLcnHvKaUdJNlVe042H2unYfhjzWTYGCw_OwVNLxNtDk1vYaF_LlCA9abpmSazK6eGzN1k5XTgWroC0ng
poetry install
Installing dependencies from lock file

Package operations: 0 installs, 5 updates, 0 removals

  - Updating platformdirs (4.2.2 -> 4.3.2)
  - Updating types-python-dateutil (2.9.0.20240821 -> 2.9.0.20240906)
  - Updating pytest (8.3.2 -> 8.3.3)
  - Updating rich (13.8.0 -> 13.8.1)
  - Updating types-requests (2.32.0.20240905 -> 2.32.0.20240907)

Installing the current project: nominal (1.0.0-beta)
poetry run pytest
=================================================================================================================== test session starts ===================================================================================================================
platform darwin -- Python 3.12.4, pytest-8.3.3, pluggy-1.5.0
rootdir: /Users/alex/nominal-io/nominal-client
configfile: pyproject.toml
testpaths: tests
plugins: anyio-4.4.0
collected 1 item                                                                                                                                                                                                                                          

tests/test_toplevel.py .                                                                                                                                                                                                                            [100%]

==================================================================================================================== 1 passed in 0.24s ====================================================================================================================
poetry run pytest tests/e2e --auth-token eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2F1dGgubm9taW5hbC50ZXN0Iiwic3ViIjoicmkuYXV0aG4uZGV2LnVzZXIuOGRlN2Q5M2MtNzU0Ny00M2Q5LWIyYjgtY2IzMGQwYzhmYTBhIiwibm9taW5hbCI6eyJ1c2VyX3V1aWQiOiI4ZGU3ZDkzYy03NTQ3LTQzZDktYjJiOC1jYjMwZDBjOGZhMGEiLCJvcmdhbml6YXRpb25fdXVpZCI6Ijc5YzJjZGQxLTRjY2QtNDI4OC1iMDZiLTI0ZTkwMmE2YjNhZiJ9LCJleHAiOjE3MjYwMzAyNzIsImF1ZCI6Imh0dHBzOi8vYXBpLm5vbWluYWwudGVzdCJ9.vQmczbf5cC4YJpPKLOMoXOwhG8CyHCN--9E3YczImSdFl3B0pULav0u6aN73PvqWVAgDYGw-Tb4XUlLH8DLkLicXVop-UN80SvseyllnMN8WdPRT6NmSVrEjKaGoTCpyRrAUCKowgEUn3owzoJACJM1TY0PhpUj8hJ5F7psKJ9DzjnqqUF2wZ_aXQ_ACu7xcOm1nC1wS1aSDnRuc8BbP38S9StfTlDwNStelzVP_QHhwKPuq0mnUY9NuziNKOXhDjHK3OOLcnHvKaUdJNlVe042H2unYfhjzWTYGCw_OwVNLxNtDk1vYaF_LlCA9abpmSazK6eGzN1k5XTgWroC0ng
=================================================================================================================== test session starts ===================================================================================================================
platform darwin -- Python 3.12.4, pytest-8.3.3, pluggy-1.5.0
rootdir: /Users/alex/nominal-io/nominal-client
configfile: pyproject.toml
plugins: anyio-4.4.0
collected 16 items                                                                                                                                                                                                                                        

tests/e2e/test_sdk.py .....                                                                                                                                                                                                                         [ 31%]
tests/e2e/test_toplevel.py ...........                                                                                                                                                                                                              [100%]

=================================================================================================================== 16 passed in 9.69s ====================================================================================================================
poetry run ruff format --check
13 files already formatted
poetry run mypy
Success: no issues found in 75 source files
poetry run ruff check --select I
All checks passed!