diff --git a/examples/README.md b/examples/README.md index 30f615bc2..38452d9d5 100644 --- a/examples/README.md +++ b/examples/README.md @@ -17,14 +17,9 @@ - Ontologies - Open In Github - Open In Colab - - - Data Rows - Open In Github - Open In Colab + Custom Embeddings + Open In Github + Open In Colab Batches @@ -32,34 +27,39 @@ Open In Colab - Projects - Open In Github - Open In Colab + User Management + Open In Github + Open In Colab - Custom Embeddings - Open In Github - Open In Colab + Basics + Open In Github + Open In Colab Data Row Metadata Open In Github Open In Colab + + Data Rows + Open In Github + Open In Colab + Quick Start Open In Github Open In Colab - Basics - Open In Github - Open In Colab + Ontologies + Open In Github + Open In Colab - User Management - Open In Github - Open In Colab + Projects + Open In Github + Open In Colab @@ -75,16 +75,16 @@ - - Composite Mask Export - Open In Github - Open In Colab - Export Data Open In Github Open In Colab + + Composite Mask Export + Open In Github + Open In Colab + Exporting to CSV Open In Github @@ -105,9 +105,14 @@ - Live Multimodal Chat Project - Open In Github - Open In Colab + Queue Management + Open In Github + Open In Colab + + + Multimodal Chat Project + Open In Github + Open In Colab Project Setup @@ -119,11 +124,6 @@ Open In Github Open In Colab - - Queue Management - Open In Github - Open In Colab - @@ -138,6 +138,11 @@ + + Conversational + Open In Github + Open In Colab + Conversational LLM Data Generation Open In Github @@ -154,19 +159,9 @@ Open In Colab - Audio - Open In Github - Open In Colab - - - Conversational - Open In Github - Open In Colab - - - PDF - Open In Github - Open In Colab + DICOM + Open In Github + Open In Colab Image @@ -174,14 +169,14 @@ Open In Colab - DICOM - Open In Github - Open In Colab + Tiled + Open In Github + Open In Colab - Conversational LLM - Open In Github - Open In Colab + Audio + Open In Github + Open In Colab HTML @@ -189,9 +184,14 @@ Open In Colab - Tiled - Open In Github - Open In Colab + Conversational LLM + Open In Github + Open In Colab + + + PDF + Open In Github + Open In Colab @@ -217,16 +217,16 @@ Open In Github Open In Colab - - Meta SAM Video - Open In Github - Open In Colab - Meta SAM Open In Github Open In Colab + + Meta SAM Video + Open In Github + Open In Colab + Import YOLOv8 Annotations Open In Github @@ -247,14 +247,9 @@ - Custom Metrics Demo - Open In Github - Open In Colab - - - Model Slices - Open In Github - Open In Colab + Model Predictions to Project + Open In Github + Open In Colab Custom Metrics Basics @@ -262,9 +257,14 @@ Open In Colab - Model Predictions to Project - Open In Github - Open In Colab + Model Slices + Open In Github + Open In Colab + + + Custom Metrics Demo + Open In Github + Open In Colab @@ -281,19 +281,9 @@ - PDF Predictions - Open In Github - Open In Colab - - - HTML Predictions - Open In Github - Open In Colab - - - Conversational Predictions - Open In Github - Open In Colab + Video Predictions + Open In Github + Open In Colab Image Predictions @@ -306,9 +296,14 @@ Open In Colab - Geospatial Predictions - Open In Github - Open In Colab + HTML Predictions + Open In Github + Open In Colab + + + Conversational Predictions + Open In Github + Open In Colab Conversational LLM Predictions @@ -316,9 +311,14 @@ Open In Colab - Video Predictions - Open In Github - Open In Colab + Geospatial Predictions + Open In Github + Open In Colab + + + PDF Predictions + Open In Github + Open In Colab diff --git a/examples/project_configuration/live_multimodal_chat_project.ipynb b/examples/project_configuration/multimodal_chat_project.ipynb similarity index 74% rename from examples/project_configuration/live_multimodal_chat_project.ipynb rename to examples/project_configuration/multimodal_chat_project.ipynb index 0eab0c809..c2f741046 100644 --- a/examples/project_configuration/live_multimodal_chat_project.ipynb +++ b/examples/project_configuration/multimodal_chat_project.ipynb @@ -16,12 +16,12 @@ "metadata": {}, "source": [ "\n", - "\n", "\n", "\n", "\n", - "\n", "" ], @@ -30,14 +30,16 @@ { "metadata": {}, "source": [ - "# Live Multimodal Chat project setup\n", + "# Multimodal chat project setup\n", "\n", - "This notebook will provide an example workflow of setting up a Live Multimodal Chat (LMC) Project with the Labelbox-Python SDK.\n", - "Live Multimodal Chat Projects are set up differently than other projects with its own unique method and modifications to existing methods:\n", + "This notebook will provide an example workflow of setting up a multimodal Chat (MMC) Project with the Labelbox-Python SDK.\n", + "Multimodal Chat Projects are set up differently than other projects with its own unique method and modifications to existing methods:\n", "\n", - "- `client.create_model_evaluation_project`: The main method used to create a Live Multimodal Chat project\n", + "- `client.create_model_evaluation_project`: The main method used to create a live multimodal Chat project.\n", + " \n", + "- `client.create_offline_model_evaluation_project`: The main method used to create a offline multimodal Chat project.\n", "\n", - "- `client.create_ontology`: Methods used to create Labelbox ontologies for LMC project this requires an `ontology_kind` parameter set to `lb.OntologyKind.ModelEvaluation`\n", + "- `client.create_ontology`: Methods used to create Labelbox ontologies for LMC project this requires an `ontology_kind` parameter set to `lb.OntologyKind.ModelEvaluation`.\n", "\n", "- `client.create_ontology_from_feature_schemas`: Similar to `client.create_ontology` but from a list of `feature schema ids` designed to allow you to use existing features instead of creating new features. This also requires an `ontology_kind` set to `lb.OntologyKind.ModelEvaluation`." ], @@ -68,7 +70,7 @@ "metadata": {}, "source": [ "## API key and client\n", - "Provide a valid API key below in order to properly connect to the Labelbox client. Please review [Create API key guide](https://docs.labelbox.com/reference/create-api-key) for more information." + "Please provide a valid API key below to connect to the Labelbox client properly. For more information, please review the [Create API key guide](https://docs.labelbox.com/reference/create-api-key)." ], "cell_type": "markdown" }, @@ -82,18 +84,18 @@ { "metadata": {}, "source": [ - "## Example: Create Live Multimodal Chat project\n", + "## Example: Create multimodal Chat project\n", "\n", - "The steps to creating a Live Multimodal Chat Project through the Labelbox-Python SDK are similar to creating a regular project. However, they vary slightly, and we will showcase the different methods in this example workflow." + "The steps to creating a multimodal Chat Projects through the Labelbox-Python SDK are similar to creating a regular project. However, they vary slightly, and we will showcase the different methods in this example workflow." ], "cell_type": "markdown" }, { "metadata": {}, "source": [ - "### Create a Live Multimodal Chat ontology\n", + "### Create a multimodal chat ontology\n", "\n", - "You can create ontologies for Model Evaluation projects the same way as creating ontologies for other projects with the only requirement of passing in a `ontology_kind` parameter which needs set to `lb.OntologyKind.ModelEvaluation`. You can create ontologies with two methods: `client.create_ontology` and `client.create_ontology_from_feature_schemas`." + "You can create ontologies for multimodal chat projects in the same way as other project ontologies using two methods: `client.create_ontology` and `client.create_ontology_from_feature_schemas`. The only additional requirement is to pass an ontology_kind parameter, which needs to be set to `lb.OntologyKind.ModelEvaluation`." ], "cell_type": "markdown" }, @@ -102,7 +104,7 @@ "source": [ "#### Option A: `client.create_ontology`\n", "\n", - "Typically, you create ontologies and generate the associated features at the same time. Below is an example of creating an ontology for your Live Multimodal Chat project using supported tools and classifications. For information on supported annotation types visit our [Live Multimodal Chat](https://docs.labelbox.com/docs/live-multimodal-chat#supported-annotation-types) guide." + "Typically, you create ontologies and generate the associated features simultaneously. Below is an example of creating an ontology for your multimodal chat project using supported tools and classifications; for information on supported annotation types, visit our [multimodal chat evaluation guide](https://docs.labelbox.com/docs/multimodal-chat#supported-annotation-types) guide." ], "cell_type": "markdown" }, @@ -128,6 +130,37 @@ "outputs": [], "execution_count": null }, + { + "metadata": {}, + "source": [ + "## Creating multimodal chat evaluation projects\n", + "\n", + "There are two versions of a multimodal chat evaluation projects:\n", + "\n", + "1. Offline multimodal chat evaluation projects: Data rows will need to be imported manually and have no live model invocation.\n", + "\n", + "2. Live multimodal chat evaluation projects: Empty data rows are generated on project creation and are filled out with live model invocation.\n", + "\n", + "We will discuss creating both types of projects with the Labelbox SDK." + ], + "cell_type": "markdown" + }, + { + "metadata": {}, + "source": [ + "## Set up offline multimodal chat evaluation project\n", + "\n", + "For an offline multimodal chat evaluation project, you must import conversational version 2 data rows. For more information, please visit our [import multimodal chat evaluation data](https://docs.labelbox.com/reference/import-multimodal-chat-data) guide. Offline multimodal chat evaluation projects are created through the SDK with `client.create_offline_model_evaluation_project`. This method uses the same parameters as `client.create_project` but provides better validation to ensure the project is set up correctly." + ], + "cell_type": "markdown" + }, + { + "metadata": {}, + "source": "project = client.create_offline_model_evaluation_project(\n name=\"\",\n description=\"\", # optional\n)", + "cell_type": "code", + "outputs": [], + "execution_count": null + }, { "metadata": {}, "source": [ @@ -224,10 +257,21 @@ { "metadata": {}, "source": [ - "**To finish setting up your LMC project, you will need to navigate to your project overview inside the Labelbox platform and select _Complete setup_ on the left side panel**" + "### Mark project setup as completed\n", + "\n", + "Once you have finalized your project and set up your model configs, you must mark the project setup as completed.\n", + "\n", + "**Once the project is marked as \"setup complete\", a user can not add, modify, or delete existing project model configs.**" ], "cell_type": "markdown" }, + { + "metadata": {}, + "source": "project.set_project_model_setup_complete()", + "cell_type": "code", + "outputs": [], + "execution_count": null + }, { "metadata": {}, "source": [