Skip to content

feat: added stringify_unsupported() #203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

SiddhantSadangi
Copy link
Member

@SiddhantSadangi SiddhantSadangi commented Apr 14, 2025

Before submitting checklist

  • Did you ask the docs owner to review all the user-facing changes?

@SiddhantSadangi SiddhantSadangi added the enhancement New feature or request label Apr 14, 2025
@SiddhantSadangi SiddhantSadangi requested a review from Copilot April 14, 2025 12:34
@SiddhantSadangi SiddhantSadangi self-assigned this Apr 14, 2025
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@SiddhantSadangi SiddhantSadangi requested a review from Copilot April 14, 2025 12:36
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (2)

src/neptune_scale/utils.py:5

  • [nitpick] The '**kwargs' parameter is defined but not used within the function. Consider removing it to simplify the function signature unless it's reserved for future extensions.
def stringify_unsupported(d: dict[str, Any], **kwargs: Any) -> dict[str, Any]:

src/neptune_scale/utils.py:44

  • [nitpick] Consider simplifying new_key assignment for clarity. For example, using: new_key = key if not prefix else f"{prefix}/{key}".
new_key = f"{prefix}/{key}" if prefix else f"{prefix}{key}"

@SiddhantSadangi SiddhantSadangi marked this pull request as ready for review April 14, 2025 13:01
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@SiddhantSadangi SiddhantSadangi requested a review from Copilot April 14, 2025 14:13
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@SiddhantSadangi SiddhantSadangi requested a review from Copilot April 14, 2025 14:22
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

@SiddhantSadangi SiddhantSadangi requested a review from Copilot April 14, 2025 14:58
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

if not isinstance(d, dict):
raise TypeError("Input must be a dictionary")

allowed_datatypes = [int, float, str, datetime, bool]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about types that are supported in some methods, but not others? E.g. File is supported in assign_files but not in log_configs. Or Histogram - supported as an element of a series, but not as an atom.

TBH I'm not entirely sure what the purpose of this function is.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is meant only to convert anything that can be represented to a primitive type, much like the original stringify_unsupported

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, what worries me here:

  1. This function does 2 unrelated things: flatten + stringify
  2. It's called stringify_unsupported yet it stringifies also types that Neptune supports. What does it even mean that Neptune doesn't support a type?

How would you feel about splitting this into 2 methods:

# just flattening dictionaries, no change to leaf values
# and no changes to the original value
def flatten(d: dict[str: Any]) -> dict[str: Any]
   pass

PRESERVED_TYPES = [int, float, str, datetime, bool]

# Assume it's a flat dictionary.
# Any value not in the `preserved_types` list gets strigified
def stringify_values(d: dict[str, Any], preserved_types: list[Type] = PRESERVED_TYPES) -> dict[str, Any]:
   pass

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does it mean that Neptune doesn't support a certain type?

image

How would you feel about splitting this into 2 methods:

I can divide it into private methods internally, but from a user's perspective, the problem remains the same. You can't pass nested dictionaries to log_configs(), so these inputs will need to be:

  1. Flattened
  2. Any unsupported types (like tensor or np.array()) will need to be converted to a supported type.

The function stringify_unsupported combines both of these operations into a single function, which reduces the boilerplate code required by the user.

I agree that the naming could be improved. I copied the name from neptune<2.x.

@normandy7 , do you have any suggestions for a better name?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

convert_to_flat_stringified_dict? Verbose, but wouldn't leave anyone guessing 😅

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might as well be flatten_and_stringify_unsupported or something like that, though. I don't think there's an obvious way to shorten it further without obfuscating it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flatten_and_cast?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't cast ambiguous, though? I'd rather use stringify.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong opinions here... Will leave it to @normandy7 and @pitercl to finalize the name :)

Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces the new function stringify_unsupported(), which flattens nested dictionaries and converts unsupported values to strings for logging purposes, along with comprehensive unit tests verifying its behavior.

  • Added stringify_unsupported() implementation in the utils module
  • Introduced multiple unit tests covering basic, collection-based, nested, mixed, edge cases, custom objects, and invalid/empty inputs

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
tests/unit/test_stringify_unsupported.py Added extensive tests covering various input types and edge cases for the new helper.
src/neptune_scale/utils.py Added the stringify_unsupported() function implementation with recursive flattening logic.

SiddhantSadangi and others added 2 commits May 12, 2025 17:46
Co-authored-by: Sabine Ståhlberg <sabine.stahlberg@neptune.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants