Skip to content

Refactor Dataset create_data_rows_sync to use upsert #1694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 13 commits into from
Jul 1, 2024
Merged

Refactor Dataset create_data_rows_sync to use upsert #1694

merged 13 commits into from
Jul 1, 2024

Conversation

vbrodsky
Copy link
Contributor

@vbrodsky vbrodsky commented Jun 25, 2024 •
edited
Loading

Description

This PR introduces improvements to the data row creation process within the SDK.
This PR refactors the following two data row creation methods:

  • Dataset.create_data_row
  • Dataset.create_data_row_sync

Both methods now leverage the upsert mutation for data row creation.

Since the upsert mutation is asynchronous, a task waiting mechanism has been implemented within these methods to ensure synchronous data row creation behavior.

A new ResourceCreationError exception has been introduced. This exception provides informative error messages in case of data creation failures, replacing the previous approach of returning None.

Stories:
https://labelbox.atlassian.net/browse/PLT-1066
https://labelbox.atlassian.net/browse/PLT-1067

On test failures:

  • User group creation tests fail, this PR should fix it
  • A couple of tests failed due to data row creation task not completing. I have verified these tests pass when run on my laptop against stage

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Document change (fix typo or modifying any markdown files, code comments or anything in the examples folder only)

All Submissions

  • Have you followed the guidelines in our Contributing document?
  • Have you provided a description?
  • Are your changes properly formatted?

New Feature Submissions

  • Does your submission pass tests?
  • Have you added thorough tests for your new feature?
  • Have you commented your code, particularly in hard-to-understand areas?
  • Have you added a Docstring?

Changes to Core Features

  • Have you written new tests for your core changes, as applicable?
  • Have you successfully run tests with your changes locally?
  • Have you updated any code comments, as applicable?

@vbrodsky vbrodsky requested a review from a team as a code owner June 25, 2024 16:03
@vbrodsky vbrodsky marked this pull request as draft June 25, 2024 16:03
@vbrodsky vbrodsky force-pushed the VB/create-data-rows-sync-use-upsert_PLT-1066 branch 6 times, most recently from 4d48571 to 9cc14e5 Compare June 26, 2024 23:33
@vbrodsky vbrodsky marked this pull request as ready for review June 27, 2024 00:34
@vbrodsky vbrodsky force-pushed the VB/create-data-rows-sync-use-upsert_PLT-1066 branch from 5ff6529 to e5ea71d Compare June 27, 2024 00:42
mrobers1982
mrobers1982 reviewed Jun 27, 2024
View reviewed changes
@vbrodsky vbrodsky force-pushed the VB/create-data-rows-sync-use-upsert_PLT-1066 branch 3 times, most recently from 4500e3a to 3ca28a0 Compare June 28, 2024 14:11
@vbrodsky vbrodsky requested a review from mrobers1982 June 28, 2024 15:37
mrobers1982
mrobers1982 reviewed Jun 28, 2024
View reviewed changes
libs/labelbox/src/labelbox/schema/dataset.py
specs = DataRowCreateItem.build(self.uid, upload_items)
task: DataUpsertTask = self._exec_upsert_data_rows(
specs, file_upload_thread_count)
task.wait_till_done()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Can this call create_data_rows to obtain a DataUpsertTask to avoid code duplication.

mrobers1982
mrobers1982 previously approved these changes Jun 28, 2024
View reviewed changes
@vbrodsky vbrodsky requested a review from mrobers1982 June 28, 2024 17:34
@vbrodsky vbrodsky force-pushed the VB/create-data-rows-sync-use-upsert_PLT-1066 branch from e4c7b5e to b5e247a Compare June 28, 2024 18:50
@vbrodsky vbrodsky merged commit 39a8b0b into develop Jul 1, 2024
12 of 22 checks passed
@vbrodsky vbrodsky deleted the VB/create-data-rows-sync-use-upsert_PLT-1066 branch July 1, 2024 13:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mrobers1982 mrobers1982 mrobers1982 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vbrodsky @mrobers1982