Skip to content

Clean up some notebooks #1669

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 11, 2024
Merged

Clean up some notebooks #1669

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3aded85
updated batches
Gabefire Jun 11, 2024
f6769d7
modified task export to remove json.loads
Gabefire Jun 11, 2024
16c2f32
Merge branch 'develop' into gu/major_fixes
Gabefire Jun 11, 2024
bfc4332
removed metadata
Gabefire Jun 11, 2024
2977be0
Merge branch 'gu/major_fixes' of github.com:Labelbox/labelbox-python …
Gabefire Jun 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion examples/annotation_import/pdf.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,7 @@
},
{
"metadata": {},
"source": "client.enable_experimental = True\ntask = lb.DataRow.export(client=client, global_keys=[global_key])\ntask.wait_till_done()\nstream = task.get_buffered_stream()\n\ntext_layer = \"\"\nfor output in stream:\n output_json = json.loads(output.json)\n text_layer = output_json[\"media_attributes\"][\"text_layer_url\"]\nprint(text_layer)",
"source": "client.enable_experimental = True\ntask = lb.DataRow.export(client=client, global_keys=[global_key])\ntask.wait_till_done()\nstream = task.get_buffered_stream()\n\ntext_layer = \"\"\nfor output in stream:\n output_json = output.json\n text_layer = output_json[\"media_attributes\"][\"text_layer_url\"]\nprint(text_layer)",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down
30 changes: 19 additions & 11 deletions examples/basics/batches.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,14 +30,15 @@
{
"metadata": {},
"source": [
"# Batches\n",
"https://docs.labelbox.com/docs/batches"
"# Batches"
],
"cell_type": "markdown"
},
{
"metadata": {},
"source": [
"This notebook covers the basics of batches:\n",
"\n",
"* A batch is collection of data rows.\n",
"* A data row cannot be part of more than one batch in a given project.\n",
"* Batches work for all data types, but there can only be one data type per project.\n",
Expand All @@ -50,7 +51,14 @@
},
{
"metadata": {},
"source": "%pip install -q \"labelbox[data]\"",
"source": [
"## Set up"
],
"cell_type": "markdown"
},
{
"metadata": {},
"source": "%pip install -q --upgrade \"labelbox[data]\"",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand All @@ -66,13 +74,13 @@
"metadata": {},
"source": [
"## API key and client\n",
"Provide a valid API key below in order to properly connect to the Labelbox Client."
"Provide a valid API key below to connect to the Labelbox client properly. For more information, please review the [Create API key](https://docs.labelbox.com/reference/create-api-key) guide."
],
"cell_type": "markdown"
},
{
"metadata": {},
"source": "# Add your API key\nAPI_KEY = None\n# To get your API key go to: Workspace settings -> API -> Create API Key\nclient = lb.Client(api_key=API_KEY)",
"source": "API_KEY = None\nclient = lb.Client(api_key=API_KEY)",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down Expand Up @@ -121,7 +129,7 @@
},
{
"metadata": {},
"source": "client.enable_experimental = True\n\nexport_task = dataset.export()\nexport_task.wait_till_done()\n\ndata_rows = []\n\n\ndef json_stream_handler(output: lb.BufferedJsonConverterOutput):\n data_row = json.loads(output.json)\n data_rows.append(data_row)\n\n\nif export_task.has_errors():\n export_task.get_buffered_stream(stream_type=lb.StreamType.ERRORS).start(\n stream_handler=lambda error: print(error))\n\nif export_task.has_result():\n export_json = export_task.get_buffered_stream(\n stream_type=lb.StreamType.RESULT).start(\n stream_handler=json_stream_handler)",
"source": "export_task = dataset.export()\nexport_task.wait_till_done()\n\ndata_rows = []\n\n\ndef json_stream_handler(output: lb.BufferedJsonConverterOutput):\n data_row = output.json\n data_rows.append(data_row)\n\n\nif export_task.has_errors():\n export_task.get_buffered_stream(stream_type=lb.StreamType.ERRORS).start(\n stream_handler=lambda error: print(error))\n\nif export_task.has_result():\n export_json = export_task.get_buffered_stream(\n stream_type=lb.StreamType.RESULT).start(\n stream_handler=json_stream_handler)",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down Expand Up @@ -247,14 +255,14 @@
},
{
"metadata": {},
"source": "client.enable_experimental = True\n\nexport_params = {\n \"attachments\": True,\n \"metadata_fields\": True,\n \"data_row_details\": True,\n \"project_details\": True,\n \"performance_details\": True,\n \"batch_ids\": [\n batch.uid\n ], # Include batch ids if you only want to export specific batches, otherwise,\n # you can export all the data without using this parameter\n}\nfilters = {}\n\n# A task is returned, this provides additional information about the status of your task, such as\n# any errors encountered\nexport_task = project.export(params=export_params, filters=filters)\nexport_task.wait_till_done()",
"source": "export_params = {\n \"attachments\": True,\n \"metadata_fields\": True,\n \"data_row_details\": True,\n \"project_details\": True,\n \"performance_details\": True,\n \"batch_ids\": [\n batch.uid\n ], # Include batch ids if you only want to export specific batches, otherwise,\n # you can export all the data without using this parameter\n}\nfilters = {}\n\n# A task is returned, this provides additional information about the status of your task, such as\n# any errors encountered\nexport_task = project.export(params=export_params, filters=filters)\nexport_task.wait_till_done()",
"cell_type": "code",
"outputs": [],
"execution_count": null
},
{
"metadata": {},
"source": "data_rows = []\n\n\ndef json_stream_handler(output: lb.BufferedJsonConverterOutput):\n data_row = json.loads(output.json)\n data_rows.append(data_row)\n\n\nif export_task.has_errors():\n export_task.get_buffered_stream(stream_type=lb.StreamType.ERRORS).start(\n stream_handler=lambda error: print(error))\n\nif export_task.has_result():\n export_json = export_task.get_buffered_stream(\n stream_type=lb.StreamType.RESULT).start(\n stream_handler=json_stream_handler)",
"source": "data_rows = []\n\n\ndef json_stream_handler(output: lb.BufferedJsonConverterOutput):\n data_row = output.json\n data_rows.append(data_row)\n\n\nif export_task.has_errors():\n export_task.get_buffered_stream(stream_type=lb.StreamType.ERRORS).start(\n stream_handler=lambda error: print(error))\n\nif export_task.has_result():\n export_json = export_task.get_buffered_stream(\n stream_type=lb.StreamType.RESULT).start(\n stream_handler=json_stream_handler)",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down Expand Up @@ -283,14 +291,14 @@
{
"metadata": {},
"source": [
"## Clean up \n",
"Uncomment and run the cell below to optionally delete the batch, dataset, and/or project created in this demo."
"## Clean up\n",
"Uncomment and run the cell below to optionally delete Labelbox objects created."
],
"cell_type": "markdown"
},
{
"metadata": {},
"source": "# Delete Batch\n# batch.delete()\n\n# Delete Project\n# project.delete()\n\n# Delete DataSet\n# dataset.delete()",
"source": "# batch.delete()\n# project.delete()\n# dataset.delete()",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down
4 changes: 2 additions & 2 deletions examples/basics/custom_embeddings.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
},
{
"metadata": {},
"source": "%pip3 install -q \"labelbox[data]\"",
"source": "%pip install -q --upgrade \"labelbox[data]\"",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down Expand Up @@ -102,7 +102,7 @@
},
{
"metadata": {},
"source": "data_rows = []\n\n\ndef json_stream_handler(output: lb.BufferedJsonConverterOutput):\n data_row = json.loads(output.json)\n data_rows.append(data_row)\n\n\nif export_task.has_errors():\n export_task.get_buffered_stream(stream_type=lb.StreamType.ERRORS).start(\n stream_handler=lambda error: print(error))\n\nif export_task.has_result():\n export_json = export_task.get_buffered_stream(\n stream_type=lb.StreamType.RESULT).start(\n stream_handler=json_stream_handler)",
"source": "data_rows = []\n\n\ndef json_stream_handler(output: lb.BufferedJsonConverterOutput):\n data_row = output.json\n data_rows.append(data_row)\n\n\nif export_task.has_errors():\n export_task.get_buffered_stream(stream_type=lb.StreamType.ERRORS).start(\n stream_handler=lambda error: print(error))\n\nif export_task.has_result():\n export_json = export_task.get_buffered_stream(\n stream_type=lb.StreamType.RESULT).start(\n stream_handler=json_stream_handler)",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down
4 changes: 2 additions & 2 deletions examples/basics/data_rows.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@
},
{
"metadata": {},
"source": "data_rows = []\n\n\ndef json_stream_handler(output: lb.BufferedJsonConverterOutput):\n data_row = json.loads(output.json)\n data_rows.append(data_row)\n\n\nif export_task.has_errors():\n export_task.get_buffered_stream(stream_type=lb.StreamType.ERRORS).start(\n stream_handler=lambda error: print(error))\n\nif export_task.has_result():\n export_json = export_task.get_buffered_stream(\n stream_type=lb.StreamType.RESULT).start(\n stream_handler=json_stream_handler)",
"source": "data_rows = []\n\n\ndef json_stream_handler(output: lb.BufferedJsonConverterOutput):\n data_row = output.json\n data_rows.append(data_row)\n\n\nif export_task.has_errors():\n export_task.get_buffered_stream(stream_type=lb.StreamType.ERRORS).start(\n stream_handler=lambda error: print(error))\n\nif export_task.has_result():\n export_json = export_task.get_buffered_stream(\n stream_type=lb.StreamType.RESULT).start(\n stream_handler=json_stream_handler)",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down Expand Up @@ -236,7 +236,7 @@
},
{
"metadata": {},
"source": "# Fetch a data row from the first dataset example\nts = dataset.export()\nts.wait_till_done()\nDATA_ROW_ID = [json.loads(output.json) for output in ts.get_buffered_stream()\n ][0][\"data_row\"][\"id\"]\nGLOBAL_KEY = [json.loads(output.json) for output in ts.get_buffered_stream()\n ][0][\"data_row\"][\"global_key\"]\n\nprint(f\"Pick either a data row id : {DATA_ROW_ID} or global key: {GLOBAL_KEY}\")",
"source": "# Fetch a data row from the first dataset example\nts = dataset.export()\nts.wait_till_done()\nDATA_ROW_ID = [output.json for output in ts.get_buffered_stream()\n ][0][\"data_row\"][\"id\"]\nGLOBAL_KEY = [output.json for output in ts.get_buffered_stream()\n ][0][\"data_row\"][\"global_key\"]\n\nprint(f\"Pick either a data row id : {DATA_ROW_ID} or global key: {GLOBAL_KEY}\")",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down
21 changes: 15 additions & 6 deletions examples/basics/projects.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@
{
"metadata": {},
"source": [
"# Projects"
"# Projects\n",
"This notebook covers the basics of projects:"
],
"cell_type": "markdown"
},
Expand All @@ -47,7 +48,14 @@
},
{
"metadata": {},
"source": "%pip install -q \"labelbox[data]\"",
"source": [
"## Set up"
],
"cell_type": "markdown"
},
{
"metadata": {},
"source": "%pip install -q --upgrade \"labelbox[data]\"",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand All @@ -62,14 +70,14 @@
{
"metadata": {},
"source": [
"# API Key and Client\n",
"Provide a valid api key below in order to properly connect to the Labelbox Client."
"## API key and client\n",
"Provide a valid API key below to connect to the Labelbox client properly. For more information, please review the [Create API key](https://docs.labelbox.com/reference/create-api-key) guide."
],
"cell_type": "markdown"
},
{
"metadata": {},
"source": "# Add your API key\nAPI_KEY = \"\"\n# To get your API key go to: Workspace settings -> API -> Create API Key\nclient = lb.Client(api_key=API_KEY)",
"source": "API_KEY = None\nclient = lb.Client(api_key=API_KEY)",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down Expand Up @@ -355,7 +363,8 @@
{
"metadata": {},
"source": [
"### Clean Up"
"## Clean up\n",
"Uncomment and run the cell below to optionally delete Labelbox objects created."
],
"cell_type": "markdown"
},
Expand Down
4 changes: 2 additions & 2 deletions examples/exports/composite_mask_export.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,7 @@
},
{
"metadata": {},
"source": "stream = export_task.get_buffered_stream()\n\nmask_tool_rgb_mapping = {}\n\nfor output in stream:\n # Parse the JSON string from the output\n output_json = json.loads(output.json)\n\n # Get the labels for the specified project ID or an empty list if the project ID is not found\n project_labels = (output_json[\"projects\"].get(PROJECT_ID,\n {}).get(\"labels\", []))\n\n # Iterate through each label\n for label in project_labels:\n # Get the list of annotations (objects) for the label\n annotations = label[\"annotations\"].get(\"objects\", [])\n\n # Iterate through each annotation\n for annotation in annotations:\n # Check if the annotation is of type \"ImageSegmentationMask\"\n if annotation.get(\"annotation_kind\") == \"ImageSegmentationMask\":\n # Add the color RGB information to the mapping dictionary\n mask_tool_rgb_mapping.setdefault(annotation[\"name\"], []).append(\n annotation[\"composite_mask\"][\"color_rgb\"])\n\nprint(mask_tool_rgb_mapping)",
"source": "stream = export_task.get_buffered_stream()\n\nmask_tool_rgb_mapping = {}\n\nfor output in stream:\n # Parse the JSON string from the output\n output_json = output.json\n\n # Get the labels for the specified project ID or an empty list if the project ID is not found\n project_labels = (output_json[\"projects\"].get(PROJECT_ID,\n {}).get(\"labels\", []))\n\n # Iterate through each label\n for label in project_labels:\n # Get the list of annotations (objects) for the label\n annotations = label[\"annotations\"].get(\"objects\", [])\n\n # Iterate through each annotation\n for annotation in annotations:\n # Check if the annotation is of type \"ImageSegmentationMask\"\n if annotation.get(\"annotation_kind\") == \"ImageSegmentationMask\":\n # Add the color RGB information to the mapping dictionary\n mask_tool_rgb_mapping.setdefault(annotation[\"name\"], []).append(\n annotation[\"composite_mask\"][\"color_rgb\"])\n\nprint(mask_tool_rgb_mapping)",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down Expand Up @@ -246,7 +246,7 @@
},
{
"metadata": {},
"source": "tools_frames_color = {}\nstream = export_task_video.get_buffered_stream()\n\n# Iterate over each output in the stream\nfor output in stream:\n output_json = json.loads(output.json)\n\n # Iterate over the labels in the specific project\n for dr in output_json[\"projects\"][VIDEO_PROJECT_ID][\"labels\"]:\n frames_data = dr[\"annotations\"][\"frames\"]\n\n # Iterate over each frame in the frames data\n for frame_key, frame_value in frames_data.items():\n\n # Iterate over each annotation in the frame\n for annotation_key, annotation_value in frame_value.items():\n if \"objects\" in annotation_key and annotation_value.values():\n\n # Iterate over each object in the annotation\n for object_key, object_value in annotation_value.items():\n if (object_value[\"annotation_kind\"] ==\n \"VideoSegmentationMask\"):\n # Update tools_frames_color with object information\n tools_frames_color.setdefault(\n object_value[\"name\"], []).append({\n frame_key:\n object_value[\"composite_mask\"]\n [\"color_rgb\"]\n })\n\nprint(tools_frames_color)",
"source": "tools_frames_color = {}\nstream = export_task_video.get_buffered_stream()\n\n# Iterate over each output in the stream\nfor output in stream:\n output_json = output.json\n\n # Iterate over the labels in the specific project\n for dr in output_json[\"projects\"][VIDEO_PROJECT_ID][\"labels\"]:\n frames_data = dr[\"annotations\"][\"frames\"]\n\n # Iterate over each frame in the frames data\n for frame_key, frame_value in frames_data.items():\n\n # Iterate over each annotation in the frame\n for annotation_key, annotation_value in frame_value.items():\n if \"objects\" in annotation_key and annotation_value.values():\n\n # Iterate over each object in the annotation\n for object_key, object_value in annotation_value.items():\n if (object_value[\"annotation_kind\"] ==\n \"VideoSegmentationMask\"):\n # Update tools_frames_color with object information\n tools_frames_color.setdefault(\n object_value[\"name\"], []).append({\n frame_key:\n object_value[\"composite_mask\"]\n [\"color_rgb\"]\n })\n\nprint(tools_frames_color)",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand Down
23 changes: 14 additions & 9 deletions examples/model_experiments/custom_metrics_demo.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,6 @@
{
"metadata": {},
"source": [
"----\n",
"\n",
"# Model Diagnostics - Custom Metrics Demo\n",
"\n",
"* Measuring model quality is critical to efficiently building models. It is important that the metrics used to measure model quality closely align with the business objectives for the model. Otherwise, slight changes in model quality, as they related to these core objectives, are lost to noise. Custom metrics enables users to measure model quality in terms of their exact business goals. By incorporating custom metrics into workflows, users can:\n",
Expand All @@ -44,13 +42,13 @@
{
"metadata": {},
"source": [
"## Environment setup"
"## Set up"
],
"cell_type": "markdown"
},
{
"metadata": {},
"source": "%pip install -q \"labelbox[data]\"",
"source": "%pip install -q --upgrade \"labelbox[data]\"",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand All @@ -65,14 +63,14 @@
{
"metadata": {},
"source": [
"## Replace with your API Key\n",
"Guides on [Create an API key](https://docs.labelbox.com/docs/create-an-api-key)"
"## API key and client\n",
"Provide a valid API key below to connect to the Labelbox client properly. For more information, please review the [Create API key](https://docs.labelbox.com/reference/create-api-key) guide."
],
"cell_type": "markdown"
},
{
"metadata": {},
"source": "API_KEY = \"\"\nclient = lb.Client(API_KEY)",
"source": "API_KEY = None\nclient = lb.Client(API_KEY)",
"cell_type": "code",
"outputs": [],
"execution_count": null
Expand All @@ -87,7 +85,14 @@
{
"metadata": {},
"source": [
"### Classification: Radio (single-choice)"
"## Classifications"
],
"cell_type": "markdown"
},
{
"metadata": {},
"source": [
"### Radio (single-choice)"
],
"cell_type": "markdown"
},
Expand All @@ -101,7 +106,7 @@
{
"metadata": {},
"source": [
"### Classification: checklist (multi-choice)"
"#### Checklist (multi-choice)"
],
"cell_type": "markdown"
},
Expand Down
Loading
Loading