Skip to content

Commit cb766ce

Browse files
author
gdj0nes
committed
ADD: video notebook
1 parent 4db26e6 commit cb766ce

File tree

1 file changed

+345
-0
lines changed

1 file changed

+345
-0
lines changed

‎examples/annotation_import/video.ipynb

Lines changed: 345 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,345 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"id": "db768cda",
6+
"metadata": {},
7+
"source": [
8+
"<td>\n",
9+
" <a target=\"_blank\" href=\"https://labelbox.com\" ><img src=\"https://labelbox.com/blog/content/images/2021/02/logo-v4.svg\" width=256/></a>\n",
10+
"</td>"
11+
]
12+
},
13+
{
14+
"cell_type": "markdown",
15+
"id": "cb5611d0",
16+
"metadata": {},
17+
"source": [
18+
"<td>\n",
19+
"<a href=\"https://colab.research.google.com/github/Labelbox/labelbox-python/blob/develop/examples/model_assisted_labeling/video_mal.ipynb\" target=\"_blank\"><img\n",
20+
"src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"></a>\n",
21+
"</td>\n",
22+
"\n",
23+
"<td>\n",
24+
"<a href=\"https://github.com/Labelbox/labelbox-python/tree/develop/examples/model_assisted_labeling/video_mal.ipynb\" target=\"_blank\"><img\n",
25+
"src=\"https://img.shields.io/badge/GitHub-100000?logo=github&logoColor=white\" alt=\"GitHub\"></a>\n",
26+
"</td>"
27+
]
28+
},
29+
{
30+
"cell_type": "markdown",
31+
"id": "stupid-court",
32+
"metadata": {},
33+
"source": [
34+
"# Video MAL"
35+
]
36+
},
37+
{
38+
"cell_type": "markdown",
39+
"id": "intellectual-idaho",
40+
"metadata": {},
41+
"source": [
42+
"* Upload model inferences for video tasks\n",
43+
"* Support types\n",
44+
" * bounding box"
45+
]
46+
},
47+
{
48+
"cell_type": "code",
49+
"execution_count": 1,
50+
"id": "voluntary-minister",
51+
"metadata": {},
52+
"outputs": [],
53+
"source": [
54+
"!pip install -q labelbox"
55+
]
56+
},
57+
{
58+
"cell_type": "code",
59+
"execution_count": 2,
60+
"id": "committed-richards",
61+
"metadata": {},
62+
"outputs": [],
63+
"source": [
64+
"import os\n",
65+
"import uuid\n",
66+
"from io import BytesIO\n",
67+
"from typing import Dict, Any, Tuple\n",
68+
"\n",
69+
"from labelbox import Client, LabelingFrontend\n",
70+
"from labelbox.schema.ontology import OntologyBuilder, Tool, Classification, Option"
71+
]
72+
},
73+
{
74+
"cell_type": "markdown",
75+
"id": "c8c876b7",
76+
"metadata": {},
77+
"source": [
78+
"# API Key and Client\n",
79+
"Provide a valid api key below in order to properly connect to the Labelbox Client."
80+
]
81+
},
82+
{
83+
"cell_type": "code",
84+
"execution_count": 5,
85+
"id": "affecting-myanmar",
86+
"metadata": {},
87+
"outputs": [],
88+
"source": [
89+
"# Add your api key\n",
90+
"API_KEY = None\n",
91+
"client = Client(api_key=API_KEY)"
92+
]
93+
},
94+
{
95+
"cell_type": "markdown",
96+
"id": "blessed-venture",
97+
"metadata": {},
98+
"source": [
99+
"### Project Setup"
100+
]
101+
},
102+
{
103+
"cell_type": "code",
104+
"execution_count": 6,
105+
"id": "suburban-crowd",
106+
"metadata": {},
107+
"outputs": [],
108+
"source": [
109+
"# We want to try out a few different tools here.\n",
110+
"ontology_builder = OntologyBuilder(\n",
111+
" tools=[Tool(tool=Tool.Type.BBOX, name=\"jellyfish\")])"
112+
]
113+
},
114+
{
115+
"cell_type": "code",
116+
"execution_count": 7,
117+
"id": "modern-program",
118+
"metadata": {},
119+
"outputs": [],
120+
"source": [
121+
"# Lets setup a project to label\n",
122+
"# Note see Ontology, Project, and Project_setup notebooks for more information on this section.\n",
123+
"project = client.create_project(name=\"video_mal_project\")\n",
124+
"dataset = client.create_dataset(name=\"video_mal_dataset\")\n",
125+
"dataset.create_data_row(\n",
126+
" row_data=\n",
127+
" \"https://storage.labelbox.com/cjhfn5y6s0pk507024nz1ocys%2Fb8837f3b-b071-98d9-645e-2e2c0302393b-jellyfish2-100-110.mp4\"\n",
128+
")\n",
129+
"editor = next(\n",
130+
" client.get_labeling_frontends(where=LabelingFrontend.name == \"Editor\"))\n",
131+
"project.setup(editor, ontology_builder.asdict())\n",
132+
"project.datasets.connect(dataset)"
133+
]
134+
},
135+
{
136+
"cell_type": "markdown",
137+
"id": "portable-grenada",
138+
"metadata": {},
139+
"source": [
140+
"#### Grab featureSchemaIds"
141+
]
142+
},
143+
{
144+
"cell_type": "code",
145+
"execution_count": 8,
146+
"id": "abstract-fifteen",
147+
"metadata": {},
148+
"outputs": [
149+
{
150+
"name": "stdout",
151+
"output_type": "stream",
152+
"text": [
153+
"{'jellyfish': 'cky3dt2lja37d0z9t26wf3qo5'}\n"
154+
]
155+
}
156+
],
157+
"source": [
158+
"# When we created a project with the ontology defined above, all of the ids were assigned.\n",
159+
"# So lets reconstruct the ontology builder with all of the ids.\n",
160+
"ontology = ontology_builder.from_project(project)\n",
161+
"# We want all of the feature schemas to be easily accessible by name.\n",
162+
"schema_lookup = {tool.name: tool.feature_schema_id for tool in ontology.tools}\n",
163+
"print(schema_lookup)"
164+
]
165+
},
166+
{
167+
"cell_type": "markdown",
168+
"id": "portuguese-arthur",
169+
"metadata": {},
170+
"source": [
171+
"## Import Format\n",
172+
"\n",
173+
"* [Documentation](https://docs.labelbox.com/docs/bounding-box-json)\n",
174+
"\n",
175+
"\n",
176+
"```\n",
177+
"Each row of the import is a unique instance\n",
178+
"\n",
179+
"schemaId: <featureSchemaId>\n",
180+
"dataRow:\n",
181+
" id: <dataRowId>\n",
182+
"Instance:\n",
183+
" [Segments]:\n",
184+
" [KeyFrames]:\n",
185+
" frame:\n",
186+
" bbox:\n",
187+
" top:\n",
188+
" bottom:\n",
189+
" height:\n",
190+
" width:\n",
191+
"```\n",
192+
"\n",
193+
"**segments**: A segment represents a continuous section where an object is visible. If an instance disappears then the segment ends. If it re-appears, a new segment is created.\n",
194+
"\n",
195+
"**keyframes**: Key frames identify the location of an instance. Between keyframes, the location of the instance is interpolated.\n",
196+
"\n",
197+
"**bbox**: The coordinates of the bounding box"
198+
]
199+
},
200+
{
201+
"cell_type": "code",
202+
"execution_count": 9,
203+
"id": "5fc417c5",
204+
"metadata": {},
205+
"outputs": [],
206+
"source": [
207+
"segments = [{\n",
208+
" \"keyframes\": [{\n",
209+
" \"frame\": 1,\n",
210+
" \"bbox\": {\n",
211+
" \"top\": 80,\n",
212+
" \"left\": 80,\n",
213+
" \"height\": 80,\n",
214+
" \"width\": 80\n",
215+
" }\n",
216+
" }, {\n",
217+
" \"frame\": 20,\n",
218+
" \"bbox\": {\n",
219+
" \"top\": 125,\n",
220+
" \"left\": 125,\n",
221+
" \"height\": 200,\n",
222+
" \"width\": 300\n",
223+
" }\n",
224+
" }]\n",
225+
"}, {\n",
226+
" \"keyframes\": [{\n",
227+
" \"frame\": 27,\n",
228+
" \"bbox\": {\n",
229+
" \"top\": 80,\n",
230+
" \"left\": 50,\n",
231+
" \"height\": 80,\n",
232+
" \"width\": 50\n",
233+
" }\n",
234+
" }]\n",
235+
"}]"
236+
]
237+
},
238+
{
239+
"cell_type": "markdown",
240+
"id": "convertible-entry",
241+
"metadata": {},
242+
"source": [
243+
"##### Create helper functions to make this much easier"
244+
]
245+
},
246+
{
247+
"cell_type": "code",
248+
"execution_count": 10,
249+
"id": "developing-beauty",
250+
"metadata": {},
251+
"outputs": [],
252+
"source": [
253+
"def create_video_bbox_ndjson(datarow_id: str, schema_id: str,\n",
254+
" segments: Dict[str, Any]) -> Dict[str, Any]:\n",
255+
" return {\n",
256+
" \"uuid\": str(uuid.uuid4()),\n",
257+
" \"schemaId\": schema_id,\n",
258+
" \"dataRow\": {\n",
259+
" \"id\": datarow_id\n",
260+
" },\n",
261+
" \"segments\": segments\n",
262+
" }"
263+
]
264+
},
265+
{
266+
"cell_type": "code",
267+
"execution_count": 11,
268+
"id": "asian-savings",
269+
"metadata": {},
270+
"outputs": [],
271+
"source": [
272+
"uploads = []\n",
273+
"\n",
274+
"for data_row in dataset.data_rows():\n",
275+
" uploads.append(\n",
276+
" create_video_bbox_ndjson(data_row.uid, schema_lookup['jellyfish'],\n",
277+
" segments))"
278+
]
279+
},
280+
{
281+
"cell_type": "markdown",
282+
"id": "perfect-seafood",
283+
"metadata": {},
284+
"source": [
285+
"### Upload the annotations"
286+
]
287+
},
288+
{
289+
"cell_type": "code",
290+
"execution_count": 12,
291+
"id": "entire-community",
292+
"metadata": {},
293+
"outputs": [],
294+
"source": [
295+
"# Let's upload!\n",
296+
"# Validate must be set to false for video bounding boxes\n",
297+
"upload_task = project.upload_annotations(name=f\"upload-job-{uuid.uuid4()}\",\n",
298+
" annotations=uploads,\n",
299+
" validate=False)"
300+
]
301+
},
302+
{
303+
"cell_type": "code",
304+
"execution_count": 13,
305+
"id": "hollywood-faculty",
306+
"metadata": {},
307+
"outputs": [
308+
{
309+
"name": "stdout",
310+
"output_type": "stream",
311+
"text": [
312+
"[]\n"
313+
]
314+
}
315+
],
316+
"source": [
317+
"# Wait for upload to finish (Will take up to five minutes)\n",
318+
"upload_task.wait_until_done()\n",
319+
"# Review the upload status\n",
320+
"print(upload_task.errors)"
321+
]
322+
}
323+
],
324+
"metadata": {
325+
"kernelspec": {
326+
"display_name": "Python 3",
327+
"language": "python",
328+
"name": "python3"
329+
},
330+
"language_info": {
331+
"codemirror_mode": {
332+
"name": "ipython",
333+
"version": 3
334+
},
335+
"file_extension": ".py",
336+
"mimetype": "text/x-python",
337+
"name": "python",
338+
"nbconvert_exporter": "python",
339+
"pygments_lexer": "ipython3",
340+
"version": "3.8.8"
341+
}
342+
},
343+
"nbformat": 4,
344+
"nbformat_minor": 5
345+
}

0 commit comments

Comments
 (0)