diff --git a/examples/README.md b/examples/README.md
index 48580ecbb..478004fe3 100644
--- a/examples/README.md
+++ b/examples/README.md
@@ -104,6 +104,11 @@
+
+ Model Chat Evaluation Project |
+  |
+  |
+
Project Setup |
 |
diff --git a/examples/project_configuration/model_chat_evaluation_project.ipynb b/examples/project_configuration/model_chat_evaluation_project.ipynb
new file mode 100644
index 000000000..ede272b78
--- /dev/null
+++ b/examples/project_configuration/model_chat_evaluation_project.ipynb
@@ -0,0 +1,197 @@
+{
+ "nbformat": 4,
+ "nbformat_minor": 2,
+ "metadata": {},
+ "cells": [
+ {
+ "metadata": {},
+ "source": [
+ "",
+ " ",
+ " | \n"
+ ],
+ "cell_type": "markdown"
+ },
+ {
+ "metadata": {},
+ "source": [
+ "\n",
+ " \n",
+ " | \n",
+ "\n",
+ "\n",
+ " \n",
+ " | "
+ ],
+ "cell_type": "markdown"
+ },
+ {
+ "metadata": {},
+ "source": [
+ "# Model Chat Evaluation Project Setup\n",
+ "\n",
+ "This notebook will provide an example workflow of setting up a Model Chat Evaluation (MCE) Project with the Labelbox-Python SDK.\n",
+ "Model Chat Evaluation Projects are set up differently than other projects with its own unique method and modifications to existing methods:\n",
+ "\n",
+ "- `client.create_model_evaluation_project`: The main method used to create a model chat evaluation project\n",
+ "\n",
+ "- `client.create_ontology`: Methods used to create Labelbox ontologies for MCE project this requires an `ontology_kind` parameter set to `lb.OntologyKind.ModelEvaluation`\n",
+ "\n",
+ "- `client.create_ontology_from_feature_schemas`: Similar to `client.create_ontology` but from a list of `feature schema ids` designed to allow you to use existing features instead of creating new features. This also requires an `ontology_kind` set to `lb.OntologyKind.ModelEvaluation`."
+ ],
+ "cell_type": "markdown"
+ },
+ {
+ "metadata": {},
+ "source": [
+ "## Set Up"
+ ],
+ "cell_type": "markdown"
+ },
+ {
+ "metadata": {},
+ "source": "%pip install -q \"labelbox[data]\"",
+ "cell_type": "code",
+ "outputs": [],
+ "execution_count": null
+ },
+ {
+ "metadata": {},
+ "source": "import labelbox as lb",
+ "cell_type": "code",
+ "outputs": [],
+ "execution_count": null
+ },
+ {
+ "metadata": {},
+ "source": [
+ "## API Key and Client\n",
+ "Provide a valid API key below in order to properly connect to the Labelbox client. Please review [Create API key guide](https://docs.labelbox.com/reference/create-api-key) for more information."
+ ],
+ "cell_type": "markdown"
+ },
+ {
+ "metadata": {},
+ "source": "API_KEY = None\nclient = lb.Client(api_key=API_KEY)",
+ "cell_type": "code",
+ "outputs": [],
+ "execution_count": null
+ },
+ {
+ "metadata": {},
+ "source": [
+ "## Example: Create Model Chat Evaluation Project\n",
+ "\n",
+ "The steps to creating a Model Chat Evaluation Project through the Labelbox-Python SDK are similar to creating a regular project. However, they vary slightly, and we will showcase the different methods in this example workflow."
+ ],
+ "cell_type": "markdown"
+ },
+ {
+ "metadata": {},
+ "source": [
+ "### Create a Model Chat Evaluation Ontology\n",
+ "\n",
+ "You can create ontologies for Model Evaluation projects the same way as creating ontologies for other projects with the only requirement of passing in a `ontology_kind` parameter which needs set to `lb.OntologyKind.ModelEvaluation`. You can create ontologies with two methods: `client.create_ontology` and `client.create_ontology_from_feature_schemas`."
+ ],
+ "cell_type": "markdown"
+ },
+ {
+ "metadata": {},
+ "source": [
+ "#### Option A: `client.create_ontology`\n",
+ "\n",
+ "Typically, you create ontologies and generate the associated features at the same time. Below is an example of creating an ontology for your model chat evaluation project using supported tools and classifications. For information on supported annotation types visit our [model chat evaluation](https://docs.labelbox.com/docs/model-chat-evaluation#supported-annotation-types) guide."
+ ],
+ "cell_type": "markdown"
+ },
+ {
+ "metadata": {},
+ "source": "ontology_builder = lb.OntologyBuilder(\n tools=[\n lb.Tool(\n tool=lb.Tool.Type.MESSAGE_SINGLE_SELECTION,\n name=\"single select feature\",\n ),\n lb.Tool(\n tool=lb.Tool.Type.MESSAGE_MULTI_SELECTION,\n name=\"multi select feature\",\n ),\n lb.Tool(tool=lb.Tool.Type.MESSAGE_RANKING, name=\"ranking feature\"),\n ],\n classifications=[\n lb.Classification(\n class_type=lb.Classification.Type.CHECKLIST,\n name=\"checklist feature\",\n options=[\n lb.Option(value=\"option 1\", label=\"option 1\"),\n lb.Option(value=\"option 2\", label=\"option 2\"),\n ],\n ),\n lb.Classification(\n class_type=lb.Classification.Type.RADIO,\n name=\"radio_question\",\n options=[\n lb.Option(value=\"first_radio_answer\"),\n lb.Option(value=\"second_radio_answer\"),\n ],\n ),\n ],\n)\n\n# Create ontology\nontology = client.create_ontology(\n \"MCE ontology\",\n ontology_builder.asdict(),\n media_type=lb.MediaType.Conversational,\n ontology_kind=lb.OntologyKind.ModelEvaluation,\n)",
+ "cell_type": "code",
+ "outputs": [],
+ "execution_count": null
+ },
+ {
+ "metadata": {},
+ "source": [
+ "### Option B: `client.create_ontology_from_feature_schemas`\n",
+ "Ontologies can also be created with feature schema IDs. This makes your ontologies with existing features compared to generating new features. You can get these features by going to the _Schema_ tab inside Labelbox. (uncomment the below code block for this option)"
+ ],
+ "cell_type": "markdown"
+ },
+ {
+ "metadata": {},
+ "source": "# ontology = client.create_ontology_from_feature_schemas(\n# \"MCE ontology\",\n# feature_schema_ids=[\"\"",
@@ -51,7 +67,7 @@
},
{
"metadata": {},
- "source": "# Add your API key\nAPI_KEY = None\n# To get your API key go to: Workspace settings -> API -> Create API Key\nclient = lb.Client(api_key=API_KEY)",
+ "source": "API_KEY = None\nclient = lb.Client(api_key=API_KEY)",
"cell_type": "code",
"outputs": [],
"execution_count": null