Skip to content

Huge optimization to annotation import library #1727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 15, 2024
Merged

Huge optimization to annotation import library #1727

merged 1 commit into from
Jul 15, 2024

Conversation

Gabefire
Copy link
Collaborator

@Gabefire Gabefire commented Jul 13, 2024 •
edited
Loading

Description

Please include a summary of the changes and the related issues. Please also include relevant motivation and context.

  • Swapped a long integration label import test for a unit test. As a comparison against staging, each of these tests can take up to 5 minutes, each now takes a few seconds.
  • Tests if a validation error was thrown while creating an object. My pr just tests the function that is used to validate with the annotation NDJSON improvements I made
  • Removed a couple of unneeded tests and will add comments on why
  • Also, this validation seems off. It validates the name of the annotation. You can still go over 5000 annotations as long as they all have different names; this is just validating there is not 5k of the same annotation on the label. Im not sure if that is intended

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

All Submissions

  • Have you followed the guidelines in our Contributing document?
  • Have you provided a description?
  • Are your changes properly formatted?

@Gabefire Gabefire requested a review from a team as a code owner July 13, 2024 04:14
Gabefire
Gabefire commented Jul 13, 2024
View reviewed changes
libs/labelbox/tests/data/annotation_import/test_bulk_import_request.py

@pytest.mark.parametrize("configured_project", [MediaType.Video], indirect = True)
@patch('labelbox.schema.annotation_import.ANNOTATION_PER_LABEL_LIMIT', 20)
def test_below_annotation_limit_on_single_data_row(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this we are constantly testing for this case everytime we do a label import

Gabefire
Gabefire commented Jul 13, 2024
View reviewed changes
libs/labelbox/tests/data/annotation_import/test_bulk_import_request.py

@pytest.mark.parametrize("configured_project", [MediaType.Video], indirect = True)
@patch('labelbox.schema.annotation_import.ANNOTATION_PER_LABEL_LIMIT', 20)
def test_above_annotation_limit_divided_among_labels_on_one_row(
Copy link
Collaborator Author

@Gabefire Gabefire Jul 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to do almost the same thing above, so it was removed. I can add this back in as a unit test instead if wanted

@Gabefire Gabefire changed the title Small optimization to annotation import library Huge optimization to annotation import library Jul 14, 2024
@Gabefire Gabefire merged commit b244ae4 into develop Jul 15, 2024
22 checks passed
@Gabefire Gabefire deleted the gu/further_annotation_optimizations branch July 15, 2024 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adrian-chang adrian-chang adrian-chang approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Gabefire @adrian-chang