From b9174a0cf6c0aea71db2c0036cdaae1e86ee1057 Mon Sep 17 00:00:00 2001 From: Mark Daoust Date: Mon, 3 Jun 2024 11:18:15 -0700 Subject: [PATCH 1/5] replace glm reference with genai.protos --- .../vectordb_with_chroma.ipynb | 1 - .../search_reranking_using_embeddings.ipynb | 9 ++-- .../docs/function-calling/python.ipynb | 54 ++++++++----------- .../tutorials/anomaly_detection.ipynb | 1 - .../clustering_with_embeddings.ipynb | 1 - .../tutorials/document_search.ipynb | 1 - .../tutorials/extract_structured_data.ipynb | 5 +- .../text_classifier_embeddings.ipynb | 1 - 8 files changed, 27 insertions(+), 46 deletions(-) diff --git a/examples/gemini/python/vectordb_with_chroma/vectordb_with_chroma.ipynb b/examples/gemini/python/vectordb_with_chroma/vectordb_with_chroma.ipynb index 8883c82c2..77f84cc5d 100644 --- a/examples/gemini/python/vectordb_with_chroma/vectordb_with_chroma.ipynb +++ b/examples/gemini/python/vectordb_with_chroma/vectordb_with_chroma.ipynb @@ -138,7 +138,6 @@ "import pandas as pd\n", "\n", "import google.generativeai as genai\n", - "import google.ai.generativelanguage as glm\n", "\n", "# Used to securely store your API key\n", "from google.colab import userdata\n", diff --git a/site/en/docs/search_reranking_using_embeddings.ipynb b/site/en/docs/search_reranking_using_embeddings.ipynb index 766331f4e..9369d058e 100644 --- a/site/en/docs/search_reranking_using_embeddings.ipynb +++ b/site/en/docs/search_reranking_using_embeddings.ipynb @@ -184,7 +184,6 @@ "import textwrap\n", "\n", "import google.generativeai as genai\n", - "import google.ai.generativelanguage as glm\n", "\n", "import wikipedia\n", "from wikipedia.exceptions import DisambiguationError, PageError\n", @@ -821,7 +820,7 @@ "In the chat history you can see all 4 steps:\n", "\n", "1. The user sent the query.\n", - "2. The model replied with a `glm.FunctionCall` calling the `wikipedia_search` with a number of relevant searches.\n", + "2. The model replied with a `genai.protos.FunctionCall` calling the `wikipedia_search` with a number of relevant searches.\n", "3. Because you set `enable_automatic_function_calling=True` when creating the `genai.ChatSession`, it executed the search function and returned the list of article summaries to the model.\n", "4. Folliwing the instructions in the prompt, the model generated a final answer based on those summaries.\n" ] @@ -1044,9 +1043,9 @@ ], "source": [ "response = chat.send_message(\n", - " glm.Content(\n", - " parts=[glm.Part(\n", - " function_response = glm.FunctionResponse(\n", + " genai.protos.Content(\n", + " parts=[genai.protos.Part(\n", + " function_response = genai.protos.FunctionResponse(\n", " name='wikipedia_search',\n", " response={'result': summaries}\n", " )\n", diff --git a/site/en/gemini-api/docs/function-calling/python.ipynb b/site/en/gemini-api/docs/function-calling/python.ipynb index ac332de0e..bd4ceff21 100644 --- a/site/en/gemini-api/docs/function-calling/python.ipynb +++ b/site/en/gemini-api/docs/function-calling/python.ipynb @@ -131,7 +131,6 @@ "import time\n", "\n", "import google.generativeai as genai\n", - "import google.ai.generativelanguage as glm\n", "\n", "from IPython import display\n", "from IPython.display import Markdown\n", @@ -206,7 +205,7 @@ "\n", "To use function calling, pass a list of functions to the `tools` parameter when creating a [`GenerativeModel`](https://ai.google.dev/api/python/google/generativeai/GenerativeModel). The model uses the function name, docstring, parameters, and parameter type annotations to decide if it needs the function to best answer a prompt.\n", "\n", - "> Important: The SDK converts function parameter type annotations to a format the API understands (`glm.FunctionDeclaration`). The API only supports a limited selection of parameter types, and the Python SDK's automatic conversion only supports a subset of that: `AllowedTypes = int | float | bool | str | list['AllowedTypes'] | dict`" + "> Important: The SDK converts function parameter type annotations to a format the API understands (`genai.protos.FunctionDeclaration`). The API only supports a limited selection of parameter types, and the Python SDK's automatic conversion only supports a subset of that: `AllowedTypes = int | float | bool | str | list['AllowedTypes'] | dict`" ] }, { @@ -327,13 +326,13 @@ "source": [ "Examine the chat history to see the flow of the conversation and how function calls are integrated within it.\n", "\n", - "The `ChatSession.history` property stores a chronological record of the conversation between the user and the Gemini model. Each turn in the conversation is represented by a [`glm.Content`](https://ai.google.dev/api/python/google/ai/generativelanguage/Content) object, which contains the following information:\n", + "The `ChatSession.history` property stores a chronological record of the conversation between the user and the Gemini model. Each turn in the conversation is represented by a [`genai.protos.Content`](https://ai.google.dev/api/python/google/ai/generativelanguage/Content) object, which contains the following information:\n", "\n", "* **Role**: Identifies whether the content originated from the \"user\" or the \"model\".\n", - "* **Parts**: A list of [`glm.Part`](https://ai.google.dev/api/python/google/ai/generativelanguage/Part) objects that represent individual components of the message. With a text-only model, these parts can be:\n", + "* **Parts**: A list of [`genai.protos.Part`](https://ai.google.dev/api/python/google/ai/generativelanguage/Part) objects that represent individual components of the message. With a text-only model, these parts can be:\n", " * **Text**: Plain text messages.\n", - " * **Function Call** ([`glm.FunctionCall`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionCall)): A request from the model to execute a specific function with provided arguments.\n", - " * **Function Response** ([`glm.FunctionResponse`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionResponse)): The result returned by the user after executing the requested function.\n", + " * **Function Call** ([`genai.protos.FunctionCall`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionCall)): A request from the model to execute a specific function with provided arguments.\n", + " * **Function Response** ([`genai.protos.FunctionResponse`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionResponse)): The result returned by the user after executing the requested function.\n", "\n", " In the previous example with the mittens calculation, the history shows the following sequence:\n", "\n", @@ -400,7 +399,7 @@ "source": [ "While this was all handled automatically, if you need more control, you can:\n", "\n", - "- Leave the default `enable_automatic_function_calling=False` and process the `glm.FunctionCall` responses yourself.\n", + "- Leave the default `enable_automatic_function_calling=False` and process the `genai.protos.FunctionCall` responses yourself.\n", "- Or use `GenerativeModel.generate_content`, where you also need to manage the chat history." ] }, @@ -541,7 +540,7 @@ "\n", "# Build the response parts.\n", "response_parts = [\n", - " glm.Part(function_response=glm.FunctionResponse(name=fn, response={\"result\": val}))\n", + " genai.protos.Part(function_response=genai.protos.FunctionResponse(name=fn, response={\"result\": val}))\n", " for fn, val in responses.items()\n", "]\n", "\n", @@ -573,17 +572,6 @@ "The `google.ai.generativelanguage` client library provides access to the low level types giving you full control." ] }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "S53E0EE8TBUF" - }, - "outputs": [], - "source": [ - "import google.ai.generativelanguage as glm" - ] - }, { "cell_type": "markdown", "metadata": { @@ -648,7 +636,7 @@ "id": "qFD4U7ym04F5" }, "source": [ - "This returns the list of `glm.Tool` objects that would be sent to the API. If the printed format is not familiar, it's because these are Google protobuf classes. Each `glm.Tool` (1 in this case) contains a list of `glm.FunctionDeclarations`, which describe a function and its arguments." + "This returns the list of `genai.protos.Tool` objects that would be sent to the API. If the printed format is not familiar, it's because these are Google protobuf classes. Each `genai.protos.Tool` (1 in this case) contains a list of `genai.protos.FunctionDeclarations`, which describe a function and its arguments." ] }, { @@ -657,7 +645,7 @@ "id": "eY6RmFQ76FVu" }, "source": [ - "Here is a declaration for the same multiply function written using the `glm` classes.\n", + "Here is a declaration for the same multiply function written using the `genai.protos` classes.\n", "\n", "Note that these classes just describe the function for the API, they don't include an implementation of it. So using this doesn't work with automatic function calling, but functions don't always need an implementation." ] @@ -670,16 +658,16 @@ }, "outputs": [], "source": [ - "calculator = glm.Tool(\n", + "calculator = genai.protos.Tool(\n", " function_declarations=[\n", - " glm.FunctionDeclaration(\n", + " genai.protos.FunctionDeclaration(\n", " name='multiply',\n", " description=\"Returns the product of two numbers.\",\n", - " parameters=glm.Schema(\n", - " type=glm.Type.OBJECT,\n", + " parameters=genai.protos.Schema(\n", + " type=genai.protos.Type.OBJECT,\n", " properties={\n", - " 'a':glm.Schema(type=glm.Type.NUMBER),\n", - " 'b':glm.Schema(type=glm.Type.NUMBER)\n", + " 'a':genai.protos.Schema(type=genai.protos.Type.NUMBER),\n", + " 'b':genai.protos.Schema(type=genai.protos.Type.NUMBER)\n", " },\n", " required=['a','b']\n", " )\n", @@ -753,7 +741,7 @@ } ], "source": [ - "glm.Tool(calculator)" + "genai.protos.Tool(calculator)" ] }, { @@ -762,7 +750,7 @@ "id": "jS6ruiTp6VBf" }, "source": [ - "Either way, you pass a representation of a `glm.Tool` or list of tools to" + "Either way, you pass a representation of a `genai.protos.Tool` or list of tools to" ] }, { @@ -787,7 +775,7 @@ "id": "517ca06297bb" }, "source": [ - "Like before the model returns a `glm.FunctionCall` invoking the calculator's `multiply` function:" + "Like before the model returns a `genai.protos.FunctionCall` invoking the calculator's `multiply` function:" ] }, { @@ -889,9 +877,9 @@ "outputs": [], "source": [ "response = chat.send_message(\n", - " glm.Content(\n", - " parts=[glm.Part(\n", - " function_response = glm.FunctionResponse(\n", + " genai.protos.Content(\n", + " parts=[genai.protos.Part(\n", + " function_response = genai.protos.FunctionResponse(\n", " name='multiply',\n", " response={'result': result}))]))" ] diff --git a/site/en/gemini-api/tutorials/anomaly_detection.ipynb b/site/en/gemini-api/tutorials/anomaly_detection.ipynb index 3efaa7cff..2722b2ac1 100644 --- a/site/en/gemini-api/tutorials/anomaly_detection.ipynb +++ b/site/en/gemini-api/tutorials/anomaly_detection.ipynb @@ -111,7 +111,6 @@ "import seaborn as sns\n", "\n", "import google.generativeai as genai\n", - "import google.ai.generativelanguage as glm\n", "\n", "# Used to securely store your API key\n", "from google.colab import userdata\n", diff --git a/site/en/gemini-api/tutorials/clustering_with_embeddings.ipynb b/site/en/gemini-api/tutorials/clustering_with_embeddings.ipynb index 4df15b305..4035164b9 100644 --- a/site/en/gemini-api/tutorials/clustering_with_embeddings.ipynb +++ b/site/en/gemini-api/tutorials/clustering_with_embeddings.ipynb @@ -112,7 +112,6 @@ "import seaborn as sns\n", "\n", "import google.generativeai as genai\n", - "import google.ai.generativelanguage as glm\n", "\n", "# Used to securely store your API key\n", "from google.colab import userdata\n", diff --git a/site/en/gemini-api/tutorials/document_search.ipynb b/site/en/gemini-api/tutorials/document_search.ipynb index c24ef9d98..49ef82660 100644 --- a/site/en/gemini-api/tutorials/document_search.ipynb +++ b/site/en/gemini-api/tutorials/document_search.ipynb @@ -109,7 +109,6 @@ "import pandas as pd\n", "\n", "import google.generativeai as genai\n", - "import google.ai.generativelanguage as glm\n", "\n", "# Used to securely store your API key\n", "from google.colab import userdata\n", diff --git a/site/en/gemini-api/tutorials/extract_structured_data.ipynb b/site/en/gemini-api/tutorials/extract_structured_data.ipynb index 1c6793399..ba09f9690 100644 --- a/site/en/gemini-api/tutorials/extract_structured_data.ipynb +++ b/site/en/gemini-api/tutorials/extract_structured_data.ipynb @@ -101,7 +101,6 @@ "import textwrap\n", "\n", "import google.generativeai as genai\n", - "import google.ai.generativelanguage as glm\n", "\n", "\n", "from IPython.display import display\n", @@ -455,7 +454,7 @@ "source": [ "If you haven't gone through the [Function calling basics](https://ai.google.dev/tutorials/function_calling_python_quickstart) tutorial yet, make sure you do that first.\n", "\n", - "With function calling your function and its parameters are described to the API as a `glm.FunctionDeclaration`. In basic cases the SDK can build the `FunctionDeclaration` from the function and its annotations. The SDK doesn't currently handle the description of nested `OBJECT` (`dict`) parameters. So you'll need to define them explicitly, for now." + "With function calling your function and its parameters are described to the API as a `genai.protos.FunctionDeclaration`. In basic cases the SDK can build the `FunctionDeclaration` from the function and its annotations. The SDK doesn't currently handle the description of nested `OBJECT` (`dict`) parameters. So you'll need to define them explicitly, for now." ] }, { @@ -701,7 +700,7 @@ "id": "kILNHmG2IED3" }, "source": [ - "The `glm.FunctionCall` class is based on Google Protocol Buffers, convert it to a more familiar JSON compatible object:" + "The `genai.protos.FunctionCall` class is based on Google Protocol Buffers, convert it to a more familiar JSON compatible object:" ] }, { diff --git a/site/en/gemini-api/tutorials/text_classifier_embeddings.ipynb b/site/en/gemini-api/tutorials/text_classifier_embeddings.ipynb index d1bcedc5a..6017fb164 100644 --- a/site/en/gemini-api/tutorials/text_classifier_embeddings.ipynb +++ b/site/en/gemini-api/tutorials/text_classifier_embeddings.ipynb @@ -111,7 +111,6 @@ "import pandas as pd\n", "\n", "import google.generativeai as genai\n", - "import google.ai.generativelanguage as glm\n", "\n", "# Used to securely store your API key\n", "from google.colab import userdata\n", From ac321cc3af30914ec68e7561399778ddea47d000 Mon Sep 17 00:00:00 2001 From: Mark Daoust Date: Mon, 3 Jun 2024 11:40:34 -0700 Subject: [PATCH 2/5] fix links --- site/en/gemini-api/docs/function-calling/python.ipynb | 8 ++++---- site/en/gemini-api/docs/get-started/python.ipynb | 4 ++-- site/en/gemini-api/docs/semantic_retrieval.ipynb | 10 +++++----- 3 files changed, 11 insertions(+), 11 deletions(-) diff --git a/site/en/gemini-api/docs/function-calling/python.ipynb b/site/en/gemini-api/docs/function-calling/python.ipynb index bd4ceff21..3151e7dd2 100644 --- a/site/en/gemini-api/docs/function-calling/python.ipynb +++ b/site/en/gemini-api/docs/function-calling/python.ipynb @@ -326,13 +326,13 @@ "source": [ "Examine the chat history to see the flow of the conversation and how function calls are integrated within it.\n", "\n", - "The `ChatSession.history` property stores a chronological record of the conversation between the user and the Gemini model. Each turn in the conversation is represented by a [`genai.protos.Content`](https://ai.google.dev/api/python/google/ai/generativelanguage/Content) object, which contains the following information:\n", + "The `ChatSession.history` property stores a chronological record of the conversation between the user and the Gemini model. Each turn in the conversation is represented by a [`genai.protos.Content`](https://ai.google.dev/api/python/google/generativeai/protos/Content) object, which contains the following information:\n", "\n", "* **Role**: Identifies whether the content originated from the \"user\" or the \"model\".\n", - "* **Parts**: A list of [`genai.protos.Part`](https://ai.google.dev/api/python/google/ai/generativelanguage/Part) objects that represent individual components of the message. With a text-only model, these parts can be:\n", + "* **Parts**: A list of [`genai.protos.Part`](https://ai.google.dev/api/python/google/generativeai/protos/Part) objects that represent individual components of the message. With a text-only model, these parts can be:\n", " * **Text**: Plain text messages.\n", - " * **Function Call** ([`genai.protos.FunctionCall`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionCall)): A request from the model to execute a specific function with provided arguments.\n", - " * **Function Response** ([`genai.protos.FunctionResponse`](https://ai.google.dev/api/python/google/ai/generativelanguage/FunctionResponse)): The result returned by the user after executing the requested function.\n", + " * **Function Call** ([`genai.protos.FunctionCall`](https://ai.google.dev/api/python/google/generativeai/protos/FunctionCall)): A request from the model to execute a specific function with provided arguments.\n", + " * **Function Response** ([`genai.protos.FunctionResponse`](https://ai.google.dev/api/python/google/generativeai/protos/FunctionResponse)): The result returned by the user after executing the requested function.\n", "\n", " In the previous example with the mittens calculation, the history shows the following sequence:\n", "\n", diff --git a/site/en/gemini-api/docs/get-started/python.ipynb b/site/en/gemini-api/docs/get-started/python.ipynb index 989cfd701..cff0564e6 100644 --- a/site/en/gemini-api/docs/get-started/python.ipynb +++ b/site/en/gemini-api/docs/get-started/python.ipynb @@ -419,7 +419,7 @@ "source": [ "Gemini can generate multiple possible responses for a single prompt. These possible responses are called `candidates`, and you can review them to select the most suitable one as the response.\n", "\n", - "View the response candidates with GenerateContentResponse.candidates:" + "View the response candidates with GenerateContentResponse.candidates:" ] }, { @@ -1488,7 +1488,7 @@ "id": "-fthdIItnqki" }, "source": [ - "Underlying the Python SDK is the google.ai.generativelanguage client library:" + "Underlying the Python SDK is the google.ai.generativelanguage client library:" ] }, { diff --git a/site/en/gemini-api/docs/semantic_retrieval.ipynb b/site/en/gemini-api/docs/semantic_retrieval.ipynb index 6da3c88e0..a9097c3dd 100644 --- a/site/en/gemini-api/docs/semantic_retrieval.ipynb +++ b/site/en/gemini-api/docs/semantic_retrieval.ipynb @@ -70,7 +70,7 @@ "\n", "A common approach used to overcome these constraints is called Retrieval Augmented Generation (RAG), which augments the prompt sent to an LLM with relevant data retrieved from an external knowledge base through an Information Retrieval (IR) mechanism. The knowledge base can be your own corpora of documents, databases, or APIs.\n", "\n", - "This notebook walks you through a workflow to improve an LLM's response by augmenting its knowledge with external text corpora and performing semantic information retrieval to answer questions using the Semantic Retriever and the Attributed Question & Answering (AQA) APIs of the [Generative Language API](https://ai.google.dev/api/python/google/ai/generativelanguage).\n", + "This notebook walks you through a workflow to improve an LLM's response by augmenting its knowledge with external text corpora and performing semantic information retrieval to answer questions using the Semantic Retriever and the Attributed Question & Answering (AQA) APIs of the Generative Language API.\n", "\n", "Note: This API is currently in [beta](https://ai.google.dev/gemini-api/docs/api-versions) and is [only available in certain regions](https://ai.google.dev/gemini-api/docs/available-regions).\n" ] @@ -719,7 +719,7 @@ "source": [ "## Attributed Question-Answering\n", "\n", - "Use the [`GenerateAnswer`](https://ai.google.dev/api/python/google/ai/generativelanguage/GenerateAnswerRequest) method to perform Attributed Question-Answering over your document, corpus, or a set of passages.\n", + "Use the [`GenerateAnswer`](https://ai.google.dev/api/python/google/generativeai/protos/GenerateAnswerRequest) method to perform Attributed Question-Answering over your document, corpus, or a set of passages.\n", "\n", "Attributed Question-Answering (AQA) refers to answering questions grounded to a given context and providing attributions(s), while minimizing hallucination.\n", "\n", @@ -769,7 +769,7 @@ "source": [ "### AQA Helpful Tips\n", "\n", - "For full API specifications, refer to the [`GenerateAnswerRequest` API Reference](https://ai.google.dev/api/python/google/ai/generativelanguage/GenerateAnswerRequest).\n", + "For full API specifications, refer to the [`GenerateAnswerRequest` API Reference](https://ai.google.dev/api/python/google/generativeai/protos/GenerateAnswerRequest).\n", "\n", "* *Passage length*: Up to 300 tokens per passage are recommended.\n", "* *Passage sorting*:\n", @@ -870,7 +870,7 @@ "source": [ "## Share the corpus\n", "\n", - "You can choose to share the corpus with others using the [`CreatePermissionRequest`](https://ai.google.dev/api/python/google/ai/generativelanguage/CreatePermissionRequest) API.\n", + "You can choose to share the corpus with others using the [`CreatePermissionRequest`](https://ai.google.dev/api/python/google/generativeai/protos/CreatePermissionRequest) API.\n", "\n", "Constraints:\n", "\n", @@ -912,7 +912,7 @@ "source": [ "## Delete the corpus\n", "\n", - "Use [`DeleteCorpusRequest`](https://ai.google.dev/api/python/google/ai/generativelanguage/DeleteCorpusRequest) to delete a user corpus and all associated `Document`s & `Chunk`s.\n", + "Use [`DeleteCorpusRequest`](https://ai.google.dev/api/python/google/generativeai/protos/DeleteCorpusRequest) to delete a user corpus and all associated `Document`s & `Chunk`s.\n", "\n", "Note that non-empty corpora will throw an error without specifying an `force=True` flag. If you set `force=True`, any `Chunk`s and objects related to this `Document` will also be deleted.\n", "\n", From 3f89dcc57ece99d05bf923bd14505e437cb4a077 Mon Sep 17 00:00:00 2001 From: Mark Daoust Date: Thu, 6 Jun 2024 17:09:40 -0700 Subject: [PATCH 3/5] Fix more glm references --- .../docs/function-calling/python.ipynb | 2 +- .../gemini-api/docs/get-started/python.ipynb | 30 +++++++++---------- 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/site/en/gemini-api/docs/function-calling/python.ipynb b/site/en/gemini-api/docs/function-calling/python.ipynb index 3151e7dd2..f07e3fec3 100644 --- a/site/en/gemini-api/docs/function-calling/python.ipynb +++ b/site/en/gemini-api/docs/function-calling/python.ipynb @@ -569,7 +569,7 @@ "AllowedType = (int | float | bool | str | list['AllowedType'] | dict[str, AllowedType]\n", "```\n", "\n", - "The `google.ai.generativelanguage` client library provides access to the low level types giving you full control." + "The `google.generativeai.protos` submodule provides access to the low level types giving you full control." ] }, { diff --git a/site/en/gemini-api/docs/get-started/python.ipynb b/site/en/gemini-api/docs/get-started/python.ipynb index cff0564e6..957d83ec8 100644 --- a/site/en/gemini-api/docs/get-started/python.ipynb +++ b/site/en/gemini-api/docs/get-started/python.ipynb @@ -957,7 +957,7 @@ "id": "AwCqtZ6D4kvk" }, "source": [ - "`glm.Content` objects contain a list of `glm.Part` objects that each contain either a text (string) or inline_data (`glm.Blob`), where a blob contains binary data and a `mime_type`. The chat history is available as a list of `glm.Content` objects in `ChatSession.history`:" + "`protos.Content` objects contain a list of `protos.Part` objects that each contain either a text (string) or inline_data (`protos.Blob`), where a blob contains binary data and a `mime_type`. The chat history is available as a list of `protos.Content` objects in `ChatSession.history`:" ] }, { @@ -1033,7 +1033,7 @@ "source": [ "## Count tokens\n", "\n", - "Large language models have a context window, and the context length is often measured in terms of the **number of tokens**. With the Gemini API, you can determine the number of tokens per any `glm.Content` object. In the simplest case, you can pass a query string to the `GenerativeModel.count_tokens` method as follows:" + "Large language models have a context window, and the context length is often measured in terms of the **number of tokens**. With the Gemini API, you can determine the number of tokens per any `protos.Content` object. In the simplest case, you can pass a query string to the `GenerativeModel.count_tokens` method as follows:" ] }, { @@ -1188,9 +1188,9 @@ "id": "zBg0eNeml3d4" }, "source": [ - "While the `genai.embed_content` function accepts simple strings or lists of strings, it is actually built around the `glm.Content` type (like GenerativeModel.generate_content). `glm.Content` objects are the primary units of conversation in the API.\n", + "While the `genai.embed_content` function accepts simple strings or lists of strings, it is actually built around the `protos.Content` type (like GenerativeModel.generate_content). `protos.Content` objects are the primary units of conversation in the API.\n", "\n", - "While the `glm.Content` object is multimodal, the `embed_content` method only supports text embeddings. This design gives the API the *possibility* to expand to multimodal embeddings." + "While the `protos.Content` object is multimodal, the `embed_content` method only supports text embeddings. This design gives the API the *possibility* to expand to multimodal embeddings." ] }, { @@ -1248,7 +1248,7 @@ "id": "jU8juHCxoUKG" }, "source": [ - "Similarly, the chat history contains a list of `glm.Content` objects, which you can pass directly to the `embed_content` function:" + "Similarly, the chat history contains a list of `protos.Content` objects, which you can pass directly to the `embed_content` function:" ] }, { @@ -1488,7 +1488,7 @@ "id": "-fthdIItnqki" }, "source": [ - "Underlying the Python SDK is the google.ai.generativelanguage client library:" + "The [`google.generativeai.protos`](https://ai.google.dev/api/python/google/generativeai/protos) submodule provides access to the low level classes used by the API behind the scenes:" ] }, { @@ -1499,7 +1499,7 @@ }, "outputs": [], "source": [ - "import google.ai.generativelanguage as glm" + "from google.generativeai import protos" ] }, { @@ -1508,10 +1508,10 @@ "id": "gm1RWcB3n_n0" }, "source": [ - "The SDK attempts to convert your message to a `glm.Content` object, which contains a list of `glm.Part` objects that each contain either:\n", + "The SDK attempts to convert your message to a `protos.Content` object, which contains a list of `protos.Part` objects that each contain either:\n", "\n", "1. a text (string)\n", - "2. `inline_data` (`glm.Blob`), where a blob contains binary `data` and a `mime_type`.\n", + "2. `inline_data` (`protos.Blob`), where a blob contains binary `data` and a `mime_type`.\n", "\n", "You can also pass any of these classes as an equivalent dictionary.\n", "\n", @@ -1530,11 +1530,11 @@ "source": [ "model = genai.GenerativeModel('gemini-1.5-flash')\n", "response = model.generate_content(\n", - " glm.Content(\n", + " protos.Content(\n", " parts = [\n", - " glm.Part(text=\"Write a short, engaging blog post based on this picture.\"),\n", - " glm.Part(\n", - " inline_data=glm.Blob(\n", + " protos.Part(text=\"Write a short, engaging blog post based on this picture.\"),\n", + " protos.Part(\n", + " inline_data=protos.Blob(\n", " mime_type='image/jpeg',\n", " data=pathlib.Path('image.jpg').read_bytes()\n", " )\n", @@ -1581,9 +1581,9 @@ "\n", "While the `genai.ChatSession` class shown earlier can handle many use cases, it does make some assumptions. If your use case doesn't fit into this chat implementation it's good to remember that `genai.ChatSession` is just a wrapper around GenerativeModel.generate_content. In addition to single requests, it can handle multi-turn conversations.\n", "\n", - "The individual messages are `glm.Content` objects or compatible dictionaries, as seen in previous sections. As a dictionary, the message requires `role` and `parts` keys. The `role` in a conversation can either be the `user`, which provides the prompts, or `model`, which provides the responses.\n", + "The individual messages are `protos.Content` objects or compatible dictionaries, as seen in previous sections. As a dictionary, the message requires `role` and `parts` keys. The `role` in a conversation can either be the `user`, which provides the prompts, or `model`, which provides the responses.\n", "\n", - "Pass a list of `glm.Content` objects and it will be treated as multi-turn chat:" + "Pass a list of `protos.Content` objects and it will be treated as multi-turn chat:" ] }, { From bd3b15910739fb0e138b1849573985f329076475 Mon Sep 17 00:00:00 2001 From: Mark Daoust Date: Thu, 6 Jun 2024 17:11:43 -0700 Subject: [PATCH 4/5] genai.protos --- .../gemini-api/docs/get-started/python.ipynb | 37 +++++++------------ 1 file changed, 13 insertions(+), 24 deletions(-) diff --git a/site/en/gemini-api/docs/get-started/python.ipynb b/site/en/gemini-api/docs/get-started/python.ipynb index 957d83ec8..c50f732f2 100644 --- a/site/en/gemini-api/docs/get-started/python.ipynb +++ b/site/en/gemini-api/docs/get-started/python.ipynb @@ -957,7 +957,7 @@ "id": "AwCqtZ6D4kvk" }, "source": [ - "`protos.Content` objects contain a list of `protos.Part` objects that each contain either a text (string) or inline_data (`protos.Blob`), where a blob contains binary data and a `mime_type`. The chat history is available as a list of `protos.Content` objects in `ChatSession.history`:" + "`genai.protos.Content` objects contain a list of `genai.protos.Part` objects that each contain either a text (string) or inline_data (`genai.protos.Blob`), where a blob contains binary data and a `mime_type`. The chat history is available as a list of `genai.protos.Content` objects in `ChatSession.history`:" ] }, { @@ -1033,7 +1033,7 @@ "source": [ "## Count tokens\n", "\n", - "Large language models have a context window, and the context length is often measured in terms of the **number of tokens**. With the Gemini API, you can determine the number of tokens per any `protos.Content` object. In the simplest case, you can pass a query string to the `GenerativeModel.count_tokens` method as follows:" + "Large language models have a context window, and the context length is often measured in terms of the **number of tokens**. With the Gemini API, you can determine the number of tokens per any `genai.protos.Content` object. In the simplest case, you can pass a query string to the `GenerativeModel.count_tokens` method as follows:" ] }, { @@ -1188,9 +1188,9 @@ "id": "zBg0eNeml3d4" }, "source": [ - "While the `genai.embed_content` function accepts simple strings or lists of strings, it is actually built around the `protos.Content` type (like GenerativeModel.generate_content). `protos.Content` objects are the primary units of conversation in the API.\n", + "While the `genai.embed_content` function accepts simple strings or lists of strings, it is actually built around the `genai.protos.Content` type (like GenerativeModel.generate_content). `genai.protos.Content` objects are the primary units of conversation in the API.\n", "\n", - "While the `protos.Content` object is multimodal, the `embed_content` method only supports text embeddings. This design gives the API the *possibility* to expand to multimodal embeddings." + "While the `genai.protos.Content` object is multimodal, the `embed_content` method only supports text embeddings. This design gives the API the *possibility* to expand to multimodal embeddings." ] }, { @@ -1248,7 +1248,7 @@ "id": "jU8juHCxoUKG" }, "source": [ - "Similarly, the chat history contains a list of `protos.Content` objects, which you can pass directly to the `embed_content` function:" + "Similarly, the chat history contains a list of `genai.protos.Content` objects, which you can pass directly to the `embed_content` function:" ] }, { @@ -1491,27 +1491,16 @@ "The [`google.generativeai.protos`](https://ai.google.dev/api/python/google/generativeai/protos) submodule provides access to the low level classes used by the API behind the scenes:" ] }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "l6aafWECnpX6" - }, - "outputs": [], - "source": [ - "from google.generativeai import protos" - ] - }, { "cell_type": "markdown", "metadata": { "id": "gm1RWcB3n_n0" }, "source": [ - "The SDK attempts to convert your message to a `protos.Content` object, which contains a list of `protos.Part` objects that each contain either:\n", + "The SDK attempts to convert your message to a `genai.protos.Content` object, which contains a list of `genai.protos.Part` objects that each contain either:\n", "\n", "1. a text (string)\n", - "2. `inline_data` (`protos.Blob`), where a blob contains binary `data` and a `mime_type`.\n", + "2. `inline_data` (`genai.protos.Blob`), where a blob contains binary `data` and a `mime_type`.\n", "\n", "You can also pass any of these classes as an equivalent dictionary.\n", "\n", @@ -1530,11 +1519,11 @@ "source": [ "model = genai.GenerativeModel('gemini-1.5-flash')\n", "response = model.generate_content(\n", - " protos.Content(\n", + " genai.protos.Content(\n", " parts = [\n", - " protos.Part(text=\"Write a short, engaging blog post based on this picture.\"),\n", - " protos.Part(\n", - " inline_data=protos.Blob(\n", + " genai.protos.Part(text=\"Write a short, engaging blog post based on this picture.\"),\n", + " genai.protos.Part(\n", + " inline_data=genai.protos.Blob(\n", " mime_type='image/jpeg',\n", " data=pathlib.Path('image.jpg').read_bytes()\n", " )\n", @@ -1581,9 +1570,9 @@ "\n", "While the `genai.ChatSession` class shown earlier can handle many use cases, it does make some assumptions. If your use case doesn't fit into this chat implementation it's good to remember that `genai.ChatSession` is just a wrapper around GenerativeModel.generate_content. In addition to single requests, it can handle multi-turn conversations.\n", "\n", - "The individual messages are `protos.Content` objects or compatible dictionaries, as seen in previous sections. As a dictionary, the message requires `role` and `parts` keys. The `role` in a conversation can either be the `user`, which provides the prompts, or `model`, which provides the responses.\n", + "The individual messages are `genai.protos.Content` objects or compatible dictionaries, as seen in previous sections. As a dictionary, the message requires `role` and `parts` keys. The `role` in a conversation can either be the `user`, which provides the prompts, or `model`, which provides the responses.\n", "\n", - "Pass a list of `protos.Content` objects and it will be treated as multi-turn chat:" + "Pass a list of `genai.protos.Content` objects and it will be treated as multi-turn chat:" ] }, { From 460cf81d8889696a50f14f08301987b8ef3e6d56 Mon Sep 17 00:00:00 2001 From: Mark Daoust Date: Thu, 6 Jun 2024 17:20:24 -0700 Subject: [PATCH 5/5] Generative Language API -> Gemini API --- site/en/gemini-api/docs/semantic_retrieval.ipynb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/site/en/gemini-api/docs/semantic_retrieval.ipynb b/site/en/gemini-api/docs/semantic_retrieval.ipynb index a9097c3dd..93a5fa1b5 100644 --- a/site/en/gemini-api/docs/semantic_retrieval.ipynb +++ b/site/en/gemini-api/docs/semantic_retrieval.ipynb @@ -70,7 +70,7 @@ "\n", "A common approach used to overcome these constraints is called Retrieval Augmented Generation (RAG), which augments the prompt sent to an LLM with relevant data retrieved from an external knowledge base through an Information Retrieval (IR) mechanism. The knowledge base can be your own corpora of documents, databases, or APIs.\n", "\n", - "This notebook walks you through a workflow to improve an LLM's response by augmenting its knowledge with external text corpora and performing semantic information retrieval to answer questions using the Semantic Retriever and the Attributed Question & Answering (AQA) APIs of the Generative Language API.\n", + "This notebook walks you through a workflow to improve an LLM's response by augmenting its knowledge with external text corpora and performing semantic information retrieval to answer questions using the Semantic Retriever and the Attributed Question & Answering (AQA) APIs of the Gemini API.\n", "\n", "Note: This API is currently in [beta](https://ai.google.dev/gemini-api/docs/api-versions) and is [only available in certain regions](https://ai.google.dev/gemini-api/docs/available-regions).\n" ] @@ -90,7 +90,7 @@ "id": "uQwqEaFLHGlL" }, "source": [ - "### Import the Generative Language API" + "### Import the Gemini API" ] }, { @@ -129,7 +129,7 @@ "Follow the steps below to setup OAuth using service accounts:\n", "\n", "\n", - "1. Enable the [Generative Language API](https://console.cloud.google.com/flows/enableapi?apiid=generativelanguage.googleapis.com){:.external}.\n", + "1. Enable the [Gemini API](https://console.cloud.google.com/flows/enableapi?apiid=generativelanguage.googleapis.com){:.external}.\n", "\n", "\n", "\n", @@ -941,7 +941,7 @@ "source": [ "## Summary and further reading\n", "\n", - "This guide introduced the Semantic Retriever and Attributed Question & Answering (AQA) APIs of the Generative Language API and showed how you can use it to perform semantic information retrieval on your custom text data. Note that this API also works with the [LlamaIndex](https://www.llamaindex.ai/){:.external} data framework. Refer to [the tutorial](https://github.com/run-llama/llama_index/blob/main/docs/docs/examples/managed/GoogleDemo.ipynb){:.external} to learn more.\n", + "This guide introduced the Semantic Retriever and Attributed Question & Answering (AQA) APIs of the Gemini API and showed how you can use it to perform semantic information retrieval on your custom text data. Note that this API also works with the [LlamaIndex](https://www.llamaindex.ai/){:.external} data framework. Refer to [the tutorial](https://github.com/run-llama/llama_index/blob/main/docs/docs/examples/managed/GoogleDemo.ipynb){:.external} to learn more.\n", "\n", "Also refer to the [API docs](https://ai.google.dev/api) to learn more about the other available functionalities.\n" ]