Skip to content

Commit 2c13433

Browse files
authored
Merge branch 'develop' into PNO/PLT-1016-Manage-external-workforces
2 parents e060f88 + 9671e60 commit 2c13433

23 files changed

+1501
-388
lines changed

.github/ISSUE_TEMPLATE/bug_report.md

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
---
2+
name: Bug Report
3+
about: Report a bug
4+
title: ''
5+
labels: 'bug-report'
6+
assignees: ''
7+
---
8+
9+
# Describe the bug
10+
A clear and concise description of what the bug is.
11+
12+
# To reproduce
13+
Steps to reproduce the behavior:
14+
1. Use method '...'
15+
2. Pass parameters '...'
16+
3. See error
17+
18+
# Expected behavior
19+
A clear and concise description of what you expected to happen.
20+
21+
# Screenshots / Videos
22+
If applicable, add screenshots or videos to help explain your problem.
23+
24+
# Environment (please complete the following information
25+
- OS: [e.g. iOS]
26+
- Python Version: [e.g. 3.8.12]
27+
- SDK + SDK Version: [e.g. labelbox@3.70.0 / labelbox[data]@3.71.0]
28+
- Python Environment [e.g. pip freeze > requirements.txt]
29+
30+
# Additional context
31+
Add any other context about the problem here.

.github/ISSUE_TEMPLATE/config.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
blank_issues_enabled: false
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
---
2+
name: New Feature Request
3+
about: Request a new feature
4+
title: ''
5+
labels: 'feature-request'
6+
assignees: ''
7+
---
8+
9+
# Is your feature request related to a problem? Please describe.
10+
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
11+
12+
# Describe the solution you'd like
13+
A clear and concise description of what you want to happen.
14+
15+
# Describe alternatives you've considered
16+
A clear and concise description of any alternative solutions or features you've considered.
17+
18+
# Additional context
19+
Add any other context or screenshots about the feature request here.

.github/pull_request_template.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Description
2+
3+
Please include a summary of the changes and the related issue. Please also include relevant motivation and context.
4+
5+
Fixes # (issue)
6+
7+
## Type of change
8+
9+
Please delete options that are not relevant.
10+
11+
- [ ] Bug fix (non-breaking change which fixes an issue)
12+
- [ ] New feature (non-breaking change which adds functionality)
13+
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
14+
- [ ] Document change (fix typo or modifying any markdown files, code comments or anything in the examples folder only)
15+
16+
## All Submissions
17+
18+
- [ ] Have you followed the guidelines in our Contributing document?
19+
- [ ] Have you provided a description?
20+
- [ ] Are your changes properly formatted?
21+
22+
## New Feature Submissions
23+
24+
- [ ] Does your submission pass tests?
25+
- [ ] Have you added thorough tests for your new feature?
26+
- [ ] Have you commented your code, particularly in hard-to-understand areas?
27+
- [ ] Have you added a Docstring?
28+
29+
## Changes to Core Features
30+
31+
- [ ] Have you written new tests for your core changes, as applicable?
32+
- [ ] Have you successfully run tests with your changes locally?
33+
- [ ] Have you updated any code comments, as applicable?

examples/README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,11 @@
4646
<td><a href="https://github.com/Labelbox/labelbox-python/tree/develop/examples/basics/data_row_metadata.ipynb" target="_blank"><img src="https://img.shields.io/badge/GitHub-100000?logo=github&logoColor=white" alt="Open In Github"></a></td>
4747
<td><a href="https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/basics/data_row_metadata.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a></td>
4848
</tr>
49+
<tr>
50+
<td>Quick Start</td>
51+
<td><a href="https://github.com/Labelbox/labelbox-python/tree/develop/examples/basics/quick_start.ipynb" target="_blank"><img src="https://img.shields.io/badge/GitHub-100000?logo=github&logoColor=white" alt="Open In Github"></a></td>
52+
<td><a href="https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/basics/quick_start.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a></td>
53+
</tr>
4954
<tr>
5055
<td>Basics</td>
5156
<td><a href="https://github.com/Labelbox/labelbox-python/tree/develop/examples/basics/basics.ipynb" target="_blank"><img src="https://img.shields.io/badge/GitHub-100000?logo=github&logoColor=white" alt="Open In Github"></a></td>
@@ -99,6 +104,11 @@
99104
</tr>
100105
</thead>
101106
<tbody>
107+
<tr>
108+
<td>Model Chat Evaluation Project</td>
109+
<td><a href="https://github.com/Labelbox/labelbox-python/tree/develop/examples/project_configuration/model_chat_evaluation_project.ipynb" target="_blank"><img src="https://img.shields.io/badge/GitHub-100000?logo=github&logoColor=white" alt="Open In Github"></a></td>
110+
<td><a href="https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/project_configuration/model_chat_evaluation_project.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a></td>
111+
</tr>
102112
<tr>
103113
<td>Project Setup</td>
104114
<td><a href="https://github.com/Labelbox/labelbox-python/tree/develop/examples/project_configuration/project_setup.ipynb" target="_blank"><img src="https://img.shields.io/badge/GitHub-100000?logo=github&logoColor=white" alt="Open In Github"></a></td>

examples/basics/projects.ipynb

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,6 +266,20 @@
266266
"outputs": [],
267267
"execution_count": null
268268
},
269+
{
270+
"metadata": {},
271+
"source": [
272+
"### Get project overview"
273+
],
274+
"cell_type": "markdown"
275+
},
276+
{
277+
"metadata": {},
278+
"source": "# Returns only the number of data rows and issues\noverview = project.get_overview()\n\n# Returns the number of data rows, issues and the details of the in_review queue\ndetailed_overview = project.get_overview(details=True)",
279+
"cell_type": "code",
280+
"outputs": [],
281+
"execution_count": null
282+
},
269283
{
270284
"metadata": {},
271285
"source": [

examples/basics/quick_start.ipynb

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
{
2+
"nbformat": 4,
3+
"nbformat_minor": 2,
4+
"metadata": {},
5+
"cells": [
6+
{
7+
"metadata": {},
8+
"source": [
9+
"<td>",
10+
" <a target=\"_blank\" href=\"https://labelbox.com\" ><img src=\"https://labelbox.com/blog/content/images/2021/02/logo-v4.svg\" width=256/></a>",
11+
"</td>\n"
12+
],
13+
"cell_type": "markdown"
14+
},
15+
{
16+
"metadata": {},
17+
"source": [
18+
"<td>\n",
19+
"<a href=\"https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/basics/quick_start.ipynb\" target=\"_blank\"><img\n",
20+
"src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"></a>\n",
21+
"</td>\n",
22+
"\n",
23+
"<td>\n",
24+
"<a href=\"https://github.com/Labelbox/labelbox-python/tree/develop/examples/basics/quick_start.ipynb\" target=\"_blank\"><img\n",
25+
"src=\"https://img.shields.io/badge/GitHub-100000?logo=github&logoColor=white\" alt=\"GitHub\"></a>\n",
26+
"</td>"
27+
],
28+
"cell_type": "markdown"
29+
},
30+
{
31+
"metadata": {},
32+
"source": [
33+
"# Quick Start\n",
34+
"\n",
35+
"This notebook is intended to be a quick overview on Labelbox-Python SDK by demonstrating a simple but common workflow.\n",
36+
"\n",
37+
"In this guide, we will be:\n",
38+
"\n",
39+
"1. Creating a dataset and importing an image data row\n",
40+
"2. Creating a ontology\n",
41+
"3. Creating a project and attaching our ontology\n",
42+
"4. Sending our data row to our project by creating a batch\n",
43+
"5. Exporting our image data row from our project\n",
44+
"\n",
45+
"This notebook is geared towards new users of Labelbox-Python SDK."
46+
],
47+
"cell_type": "markdown"
48+
},
49+
{
50+
"metadata": {},
51+
"source": [
52+
"## Setup\n",
53+
"\n",
54+
"We first need to install the `labelbox` library and then import the SDK module. It is recommended to install `\"labelbox[data]\"` over `labelbox` to obtain all the correct dependencies. We will also be importing the Python `uuid` library to generate universal unique IDs for the variety of objects that will be created with this notebook."
55+
],
56+
"cell_type": "markdown"
57+
},
58+
{
59+
"metadata": {},
60+
"source": "%pip install -q \"labelbox[data]\"",
61+
"cell_type": "code",
62+
"outputs": [],
63+
"execution_count": null
64+
},
65+
{
66+
"metadata": {},
67+
"source": "import labelbox as lb\nimport uuid",
68+
"cell_type": "code",
69+
"outputs": [],
70+
"execution_count": null
71+
},
72+
{
73+
"metadata": {},
74+
"source": [
75+
"## API Key and Client\n",
76+
"Provide a valid API key below to connect to the Labelbox client properly. For more information, please review the [Create API Key](https://docs.labelbox.com/reference/create-api-key) guide."
77+
],
78+
"cell_type": "markdown"
79+
},
80+
{
81+
"metadata": {},
82+
"source": "API_KEY = None\nclient = lb.Client(api_key=API_KEY)",
83+
"cell_type": "code",
84+
"outputs": [],
85+
"execution_count": null
86+
},
87+
{
88+
"metadata": {},
89+
"source": [
90+
"## Step 1: Create Dataset and Import Data Row\n",
91+
"\n",
92+
"Below, we will create a dataset and then attach a publicly hosted image data row. Typically, you would either import data rows hosted on a cloud provider (_recommended_) or import them locally. For more information, visit our [import image data section](https://docs.labelbox.com/reference/image) in our developer guides.\n",
93+
"\n",
94+
"- Data rows are internal representations of an asset in Labelbox. A data row contains the asset to be labeled and all of the relevant information about that asset\n",
95+
"- A dataset is a collection of data rows imported into Labelbox. They live inside the [_Catalog_](https://docs.labelbox.com/docs/catalog-overview) section of Labelbox."
96+
],
97+
"cell_type": "markdown"
98+
},
99+
{
100+
"metadata": {},
101+
"source": "# Create dataset from client\ndataset = client.create_dataset(name=\"Quick Start Example Dataset\")\n\nglobal_key = str(uuid.uuid4()) # Unique user specified ID\n\n# Data row structure\nimage_data_rows = [{\n \"row_data\":\n \"https://storage.googleapis.com/labelbox-datasets/image_sample_data/2560px-Kitano_Street_Kobe01s5s4110.jpeg\",\n \"global_key\":\n global_key,\n \"media_type\":\n \"IMAGE\",\n}]\n\n# Bulk import data row\ntask = dataset.create_data_rows(image_data_rows) # List of data rows\ntask.wait_till_done()\nprint(task.errors) # Print any errors",
102+
"cell_type": "code",
103+
"outputs": [],
104+
"execution_count": null
105+
},
106+
{
107+
"metadata": {},
108+
"source": [
109+
"## Step 2: Creating an Ontology\n",
110+
"\n",
111+
"Before we send our data row to a labeling project we first must create an ontology. In the example below we will be creating a simple ontology with a bounding box tool and a checklist classification feature. For more information, visit the [ontology section](https://docs.labelbox.com/reference/ontology) inside our developer guides. \n",
112+
"\n",
113+
"* An ontology is a collection of annotations and their relationships (also known as a taxonomy). Ontologies can be reused across different projects. It is essential for data labeling, model training, and evaluation. Created ontologies with there associated features are located inside the _Schema_ section within Labelbox."
114+
],
115+
"cell_type": "markdown"
116+
},
117+
{
118+
"metadata": {},
119+
"source": "# Bounding box feature\nobject_features = [\n lb.Tool(\n tool=lb.Tool.Type.BBOX,\n name=\"regulatory-sign\",\n color=\"#ff0000\",\n )\n]\n\n# Checklist feature\nclassification_features = [\n lb.Classification(\n class_type=lb.Classification.Type.CHECKLIST,\n name=\"Quality Issues\",\n options=[\n lb.Option(value=\"blurry\", label=\"Blurry\"),\n lb.Option(value=\"distorted\", label=\"Distorted\"),\n ],\n )\n]\n\n# Builder function\nontology_builder = lb.OntologyBuilder(tools=object_features,\n classifications=classification_features)\n\n# Create ontology\nontology = client.create_ontology(\n \"Ontology from new features\",\n ontology_builder.asdict(),\n media_type=lb.MediaType.Image,\n)",
120+
"cell_type": "code",
121+
"outputs": [],
122+
"execution_count": null
123+
},
124+
{
125+
"metadata": {},
126+
"source": [
127+
"## Step 3: Creating a Project and Attaching our Ontology\n",
128+
"\n",
129+
"Now that we have made our ontology, we are ready to create a project where we can label our data row.\n",
130+
"\n",
131+
"* Projects are labeling environments in Labelbox similar to a factory assembly line for producing annotations. The initial state of the project can start with raw data, pre-existing ground truth, or pre-labeled data."
132+
],
133+
"cell_type": "markdown"
134+
},
135+
{
136+
"metadata": {},
137+
"source": "# Create a new project\nproject = client.create_project(\n name=\"Quick Start Example Project\",\n media_type=lb.MediaType.Image,\n)\n\n# Attach created ontology\nproject.setup_editor(ontology)",
138+
"cell_type": "code",
139+
"outputs": [],
140+
"execution_count": null
141+
},
142+
{
143+
"metadata": {},
144+
"source": [
145+
"## Step 4: Sending our Data Row to our Project by Creating a Batch\n",
146+
"\n",
147+
"With our project created, we can send our data rows by creating a batch. Our data rows will start in the initial labeling queue, where labelers are able to annotate our data row.\n",
148+
"\n",
149+
"* A batch is a curated selection of data rows you can send to a project for labeling. You can create a batch with a combination of data rows within any dataset. For more information on creating batches, review the [batches section](https://docs.labelbox.com/reference/batch#create-a-batch) of our developer guides."
150+
],
151+
"cell_type": "markdown"
152+
},
153+
{
154+
"metadata": {},
155+
"source": "project.create_batch(\n name=\"Quick Start Example Batch\" + str(uuid.uuid4()),\n global_keys=[\n global_key\n ], # Global key we used earlier in this guide to create our dataset\n priority=5,\n)",
156+
"cell_type": "code",
157+
"outputs": [],
158+
"execution_count": null
159+
},
160+
{
161+
"metadata": {},
162+
"source": [
163+
"## Step 5: Exporting from our Project\n",
164+
"\n",
165+
"We have now successfully set up a project for labeling using only the SDK! \ud83d\ude80 \n",
166+
"\n",
167+
"From here, you can either label our data row directly inside the [labeling queue](https://docs.labelbox.com/docs/labeling-queue) or [import annotations](https://docs.labelbox.com/reference/import-image-annotations) directly through our SDK. Below we will demonstrate the final step of this guide by exporting from our project. Since we did not label any data rows or import annotations within this guide, no labels will be presented on our data row. For a full overview of exporting, visit our [export overview](https://docs.labelbox.com/reference/label-export) developer guide."
168+
],
169+
"cell_type": "markdown"
170+
},
171+
{
172+
"metadata": {},
173+
"source": "# Start export from project\nexport_task = project.export()\nexport_task.wait_till_done()\n\n# Conditional if task has errors\nif export_task.has_errors():\n export_task.get_buffered_stream(stream_type=lb.StreamType.ERRORS).start(\n stream_handler=lambda error: print(error))\n\n# Start export stream\nstream = export_task.get_buffered_stream()\n\n# Iterate through data rows\nfor data_row in stream:\n print(data_row.json)",
174+
"cell_type": "code",
175+
"outputs": [],
176+
"execution_count": null
177+
},
178+
{
179+
"metadata": {},
180+
"source": [
181+
"## Clean Up\n",
182+
"\n",
183+
"This section serves as an optional clean-up step to delete the Labelbox assets created within this guide. You will need to uncomment the delete methods shown."
184+
],
185+
"cell_type": "markdown"
186+
},
187+
{
188+
"metadata": {},
189+
"source": "# project.delete()\n# client.delete_unused_ontology(ontology.uid)\n# dataset.delete()",
190+
"cell_type": "code",
191+
"outputs": [],
192+
"execution_count": null
193+
}
194+
]
195+
}

0 commit comments

Comments
 (0)