Skip to content

MODEL-1489: Allow marking Label with "is_benchmark_reference" flag #1718

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 17 commits into from
Jul 31, 2024
Merged

MODEL-1489: Allow marking Label with "is_benchmark_reference" flag #1718

merged 17 commits into from
Jul 31, 2024

Conversation

dubininsergey
Copy link
Contributor

@dubininsergey dubininsergey commented Jul 8, 2024 •
edited
Loading

Description

This PR enables marking import labels with "is_benchmark_reference" flag. Such labels will be set as benchmark references if the target project support "benchmark" quality mode.

https://www.loom.com/share/ccca6e59420b4848b5f146c9fdbe9950

Fixes # (issue)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Document change (fix typo or modifying any markdown files, code comments or anything in the examples folder only)

All Submissions

  • Have you followed the guidelines in our Contributing document?
  • Have you provided a description?
  • Are your changes properly formatted?

New Feature Submissions

  • Does your submission pass tests?
  • Have you added thorough tests for your new feature?
  • Have you commented your code, particularly in hard-to-understand areas?
  • Have you added a Docstring?

Changes to Core Features

  • Have you written new tests for your core changes, as applicable?
  • Have you successfully run tests with your changes locally?
  • Have you updated any code comments, as applicable?

@dubininsergey dubininsergey requested a review from a team as a code owner July 8, 2024 11:28
@dubininsergey dubininsergey added hold do not review yet! do not merge labels Jul 8, 2024
@dubininsergey
Copy link
Contributor Author

The PR is awaiting for the complementary changes in the BE to be merged/deployed

@dubininsergey dubininsergey force-pushed the sdubinin/model-1489 branch from 29deb0c to d1173a4 Compare July 9, 2024 07:09
@review-notebook-app Review Notebook App
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@dubininsergey dubininsergey changed the base branch from develop to kozikkam/MODEL-1474 July 9, 2024 07:11
Base automatically changed from kozikkam/MODEL-1474 to develop July 24, 2024 17:17
@dubininsergey dubininsergey requested a review from vbrodsky July 30, 2024 08:05
@dubininsergey dubininsergey removed the hold do not review yet! label Jul 30, 2024
vbrodsky
vbrodsky reviewed Jul 30, 2024
View reviewed changes
libs/labelbox/src/labelbox/data/serialization/ndjson/converter.py
by_alias=True,
exclude={"uuid"} if annotation_uuid == "None" else None,
)
for k, v in list(res.items()):
if k in IGNORE_IF_NONE and v is None:
del res[k]
if getattr(label, 'is_benchmark_reference'):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you try to declare class Label(_CamelCaseMixin) , I do not think you would need this line

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to avoid juggling with mixins because this caused MRO problems in the past. Do you think we can leave it as is?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the way I have it above should prob work, can you try? If it does not work (i.e. test fails etc) we could use your approach, but I rather not have custom serialization if we can avoid it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, the logic is the following: if there is is_benchmark_reference attribute set for the label, then all child annotations should have isBenchmarkReferenceLabel set to True. I don't see how the code above would help to avoid this logic

libs/labelbox/tests/data/annotation_import/test_data_types.py

from labelbox.data.annotation_types.data import (
AudioData,
ConversationData,
DicomData,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why was this removed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the imports were unused

@dubininsergey dubininsergey requested a review from vbrodsky July 30, 2024 17:18
@vbrodsky vbrodsky merged commit 4e78990 into develop Jul 31, 2024
22 of 27 checks passed
@vbrodsky vbrodsky deleted the sdubinin/model-1489 branch July 31, 2024 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vbrodsky vbrodsky vbrodsky approved these changes

@tomislav-peharda tomislav-peharda tomislav-peharda left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@dubininsergey @vbrodsky @tomislav-peharda @kozikkamil