-
Notifications
You must be signed in to change notification settings - Fork 1
feat: added stringify_unsupported() #203
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (2)
src/neptune_scale/utils.py:5
- [nitpick] The '**kwargs' parameter is defined but not used within the function. Consider removing it to simplify the function signature unless it's reserved for future extensions.
def stringify_unsupported(d: dict[str, Any], **kwargs: Any) -> dict[str, Any]:
src/neptune_scale/utils.py:44
- [nitpick] Consider simplifying new_key assignment for clarity. For example, using: new_key = key if not prefix else f"{prefix}/{key}".
new_key = f"{prefix}/{key}" if prefix else f"{prefix}{key}"
…-ai/neptune-client-scale into ss/stringify_unsupported
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
This reverts commit eff02c6.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
if not isinstance(d, dict): | ||
raise TypeError("Input must be a dictionary") | ||
|
||
allowed_datatypes = [int, float, str, datetime, bool] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about types that are supported in some methods, but not others? E.g. File
is supported in assign_files
but not in log_configs
. Or Histogram
- supported as an element of a series, but not as an atom.
TBH I'm not entirely sure what the purpose of this function is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is meant only to convert anything that can be represented to a primitive type, much like the original stringify_unsupported
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, what worries me here:
- This function does 2 unrelated things: flatten + stringify
- It's called
stringify_unsupported
yet it stringifies also types that Neptune supports. What does it even mean that Neptune doesn't support a type?
How would you feel about splitting this into 2 methods:
# just flattening dictionaries, no change to leaf values
# and no changes to the original value
def flatten(d: dict[str: Any]) -> dict[str: Any]
pass
PRESERVED_TYPES = [int, float, str, datetime, bool]
# Assume it's a flat dictionary.
# Any value not in the `preserved_types` list gets strigified
def stringify_values(d: dict[str, Any], preserved_types: list[Type] = PRESERVED_TYPES) -> dict[str, Any]:
pass
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does it mean that Neptune doesn't support a certain type?
How would you feel about splitting this into 2 methods:
I can divide it into private methods internally, but from a user's perspective, the problem remains the same. You can't pass nested dictionaries to log_configs()
, so these inputs will need to be:
- Flattened
- Any unsupported types (like
tensor
ornp.array()
) will need to be converted to a supported type.
The function stringify_unsupported
combines both of these operations into a single function, which reduces the boilerplate code required by the user.
I agree that the naming could be improved. I copied the name from neptune<2.x
.
@normandy7 , do you have any suggestions for a better name?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
convert_to_flat_stringified_dict
? Verbose, but wouldn't leave anyone guessing 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might as well be flatten_and_stringify_unsupported
or something like that, though. I don't think there's an obvious way to shorten it further without obfuscating it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
flatten_and_cast
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't cast
ambiguous, though? I'd rather use stringify
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No strong opinions here... Will leave it to @normandy7 and @pitercl to finalize the name :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces the new function stringify_unsupported(), which flattens nested dictionaries and converts unsupported values to strings for logging purposes, along with comprehensive unit tests verifying its behavior.
- Added stringify_unsupported() implementation in the utils module
- Introduced multiple unit tests covering basic, collection-based, nested, mixed, edge cases, custom objects, and invalid/empty inputs
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
File | Description |
---|---|
tests/unit/test_stringify_unsupported.py | Added extensive tests covering various input types and edge cases for the new helper. |
src/neptune_scale/utils.py | Added the stringify_unsupported() function implementation with recursive flattening logic. |
Co-authored-by: Sabine Ståhlberg <sabine.stahlberg@neptune.ai>
Before submitting checklist