Skip to content

[MODEL-1448] Upsert label feedback method #1684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 24, 2024
Merged

Conversation

dubininsergey
Copy link
Contributor

Description

Introduced a method to submit label feedback which includes a free-text message and numeric scores.

Fixes # (issue)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Document change (fix typo or modifying any markdown files, code comments or anything in the examples folder only)

All Submissions

  • Have you followed the guidelines in our Contributing document?
  • Have you provided a description?
  • Are your changes properly formatted?

New Feature Submissions

  • Does your submission pass tests?
  • Have you added thorough tests for your new feature?
  • Have you commented your code, particularly in hard-to-understand areas?
  • Have you added a Docstring?

Changes to Core Features

  • Have you written new tests for your core changes, as applicable?
  • Have you successfully run tests with your changes locally?
  • Have you updated any code comments, as applicable?

@dubininsergey dubininsergey requested a review from a team as a code owner June 20, 2024 12:52
@dubininsergey dubininsergey requested review from whistler and adrian-chang and removed request for a team June 20, 2024 12:52
@@ -2216,3 +2216,27 @@ def get_embedding_by_name(self, name: str) -> Embedding:
return e
raise labelbox.exceptions.ResourceNotFoundError(Embedding,
dict(name=name))

def upsert_label_feedback(self, label_id: str, feedback: str,
scores: Dict[str, float]) -> Entity.Label:
Copy link
Collaborator

@mihhail-m mihhail-m Jun 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return type is Entity.Label, but actual type is List[Entity.Label]. Probably its better to match the actual returned type

Returns: A list of LabelScore instances

"""
mutation_str = """mutation UpsertAutoQaLabelFeedbackPyApi($labelId: ID!, $feedback: String!, $scores: Json!){
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to keep mutation string formatted to keep style consistency across the file

def upsert_label_feedback(self, label_id: str, feedback: str,
scores: Dict[str, float]) -> Entity.Label:
"""

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra whitespace

Comment on lines 2227 to 2228
scores: A dict of scores, the key is a score name and the value is the score value
Returns: A list of LabelScore instances
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

white space between to keep style consistent with other methods

feedback: Free text comment regarding the label
scores: A dict of scores, the key is a score name and the value is the score value
Returns: A list of LabelScore instances

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra whitespace

Comment on lines 6 to 13
"""
a label score

Attributes
name
score

"""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice to preserve the same docs style format as with the rest of schema objects. Look Dataset, Batch for examples.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand this class correctly, this is just a data class.
My personal opinion you should use a pydantic model instead of DbObject as a parent here. DbObject is our home-grown graphql client framework. Unless your class executes queries using this framework, no need to use it

Comment on lines 6 to 13
"""
a label score

Attributes
name
score

"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand this class correctly, this is just a data class.
My personal opinion you should use a pydantic model instead of DbObject as a parent here. DbObject is our home-grown graphql client framework. Unless your class executes queries using this framework, no need to use it

@@ -0,0 +1,15 @@
from labelbox import pydantic_compat
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a new package / class we also need to make sure it will appear in readthedocs by adding a new .rst in docs/labelbox/ file and updating docs/labelbox/index.rst
FYI the same is documented in CONTRIBUTING.md under General Guidelines

A list of LabelScore instances
"""
mutation_str = """
mutation UpsertAutoQaLabelFeedbackPyApi(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this our public doc (on readthedocs) please do not internal mutation code here

can you elaborate?

Rather, I would recommend to add some method usages

will add a usage example now 👍

@manuaero manuaero merged commit 97fd09f into develop Jun 24, 2024
11 of 20 checks passed
@manuaero manuaero deleted the sdubinin/model-1448 branch June 24, 2024 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants