Skip to content

feat: cleanup & simplify timestamp handling #54

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 54 commits into from
Oct 1, 2024
Merged

Conversation

alkasm
Copy link
Contributor

@alkasm alkasm commented Sep 23, 2024

I went through a few different iterations to find a specification that worked well. What I ended up with wasn't too far from where we started, so I opted for backcompat with warnings for changes to how we're handling relative timestamps.

_AnyTimeDomain: all time domains that we allow

This is value that should be used in the high-level client. It allows for basic literals + strongly typed values.

_LiteralAbsolute values

These are the values we expect many users will use - simple, literal strings.

"iso_8601"
"epoch_nanoseconds"
"epoch_microseconds"
"epoch_milliseconds"
"epoch_seconds"
"epoch_minutes"
"epoch_hours"

TypedTimeDomain values

These are the strongly typed time domains. These include additional parameterized types, like Relative and Custom.

nm.ts.Iso8601()
nm.ts.Epoch("nanoseconds")
nm.ts.Epoch("microseconds")
nm.ts.Epoch("milliseconds")
nm.ts.Epoch("seconds")
nm.ts.Epoch("minutes")
nm.ts.Epoch("hours")
nm.ts.Relative("nanoseconds", start=datetime(2024, 9, 23, 18, 12, 0))
nm.ts.Relative("microseconds", start=datetime(2024, 9, 23, 18, 12, 0))
nm.ts.Relative("milliseconds", start=datetime(2024, 9, 23, 18, 12, 0))
nm.ts.Relative("seconds", start=datetime(2024, 9, 23, 18, 12, 0))
nm.ts.Relative("minutes", start=datetime(2024, 9, 23, 18, 12, 0))
nm.ts.Relative("hours", start=datetime(2024, 9, 23, 18, 12, 0))
nm.ts.Custom(r"yyyy-MM-dd[T]hh:mm:ss")
nm.ts.Custom(r"MM-dd[T]hh:mm:ss", default_year=2024)

TypedTimeDomain constants

These are equivalent to the string representations.

nm.ts.ISO_8601
nm.ts.EPOCH_NANOSECONDS
nm.ts.EPOCH_MICROSECONDS
nm.ts.EPOCH_MILLISECONDS
nm.ts.EPOCH_SECONDS
nm.ts.EPOCH_MINUTES
nm.ts.EPOCH_HOURS

_LiteralRelativeDeprecated values

Usage of these values is deprecated and will send a UserWarning to use nm.ts.Relative(). These are deprecated since we want the users to explicitly add the start time. However these are currently in-use, so we're emitting a warning but still allowing them for now.

"relative_nanoseconds"
"relative_microseconds"
"relative_milliseconds"
"relative_seconds"
"relative_minutes"
"relative_hours"

The warning raised will look like:

/Users/alex/nominal-io/nominal-client/nominal/timedomain.py:87: UserWarning: specifying 'relative_{unit}' as a string is deprecated and will be removed in a future version: use `nm.timedomain.Relative` instead. for example: instead of 'relative_seconds', use `nm.timedomain.Relative('seconds', start=datetime.now())`. 

Exploration

winner: mix of strings + types, but advertise with constants

dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.ISO_8601)
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.ABSOLUTE_NANOSECONDS)
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.Relative("nanoseconds", offset=15))
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.Custom(r"yyyy-MM-dd[T]hh:mm:ss"))

current: mix of strings + types: relative offsets not supported yet, would need to create a new type for it

dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", "iso_8601")
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", "epoch_nanoseconds")
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", "relative_nanoseconds")
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.CustomTimestampFormat(r"yyyy-MM-dd[T]hh:mm:ss"))
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.RelativeTimestampFormat("nanoseconds", offset=15))

strongly typed: types and docstrings are extremely clear, allows more flexibility per-type

dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.Iso8601())
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.Absolute("ns"))
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.Relative("nanoseconds", offset=15))
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.Custom(r"yyyy-MM-dd[T]hh:mm:ss"))

flexible factory function: multiple behaviors depending on how you call it; difficult to document but easy to read

dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain("iso_8601"))
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain("absolute", "ns"))
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain("relative", "ns", offset=15))
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain("custom", r"yyyy-MM-dd[T]hh:mm:ss"))

mix of strings + types, but advertise with singletons

dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.iso_8601)
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.absolute_nanoseconds)
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.Relative("nanoseconds", offset=15))
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.Custom(r"yyyy-MM-dd[T]hh:mm:ss"))

factory functions for each type - pretty chill but wordier than strings

dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.iso_8601())
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.absolute_nanoseconds())
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.relative_nanoseconds(offset=15))
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.custom(r"yyyy-MM-dd[T]hh:mm:ss"))

singletons (with transforming class methods) - too magic, custom() behaves differently

dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.iso_8601)
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.absolute_nanoseconds)
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.relative_nanoseconds)
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.relative_nanoseconds.offset(15))
dataset = upload_csv("path/to/file.csv", "dataset", "timestamp", nm.timedomain.custom(r"yyyy-MM-dd[T]hh:mm:ss"))

@alkasm
Copy link
Contributor Author

alkasm commented Sep 24, 2024

I liked timedomain for the module name, but it's long / feels verbose. Can't use time since it's a built-in, and domain feels too generic. chrono or clock could work, but a bit loaded due to cpp. @alxhill suggested ts which is probably sufficient, so going with that.

Copy link
Contributor

@alxhill alxhill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good overall, happy to take a closer a look at any specifics if you want a line-by-line

@alkasm alkasm merged commit 2211070 into main Oct 1, 2024
5 checks passed
@alkasm alkasm deleted the alkasm/the-good-times branch October 1, 2024 00:11
This was referenced Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants