diff --git a/README.md b/README.md index 1bf3babd9..f25118589 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,9 @@ These are the source files for the guide and tutorials on the [Generative AI developer site](https://ai.google.dev/), home to the Gemini API and Gemma. +Note: Much of the example content here has moved to the [Gemini API +Cookbook](https://github.com/google-gemini/). + | Path | Description | | ---- | ----------- | | [`site/`](site/) | Notebooks and other content used directly on ai.google.dev. | diff --git a/examples/gemini/python/langchain/Gemini_LangChain_QA_Chroma_WebLoad.ipynb b/examples/gemini/python/langchain/Gemini_LangChain_QA_Chroma_WebLoad.ipynb deleted file mode 100644 index 6d2342a40..000000000 --- a/examples/gemini/python/langchain/Gemini_LangChain_QA_Chroma_WebLoad.ipynb +++ /dev/null @@ -1,684 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "Tce3stUlHN0L" - }, - "source": [ - "##### Copyright 2024 Google LLC." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "tuOe1ymfHZPu" - }, - "outputs": [], - "source": [ - "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", - "# you may not use this file except in compliance with the License.\n", - "# You may obtain a copy of the License at\n", - "#\n", - "# https://www.apache.org/licenses/LICENSE-2.0\n", - "#\n", - "# Unless required by applicable law or agreed to in writing, software\n", - "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", - "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", - "# See the License for the specific language governing permissions and\n", - "# limitations under the License." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "0c5ea3f4a75c" - }, - "source": [ - "# Question Answering using Gemini, LangChain, and Chroma" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "awKO767lQIWh" - }, - "source": [ - "\n", - " \n", - " \n", - "
\n", - " Run in Google Colab\n", - " \n", - " View source on GitHub\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "479790a71f3c" - }, - "source": [ - "## Overview\n", - "\n", - "[Gemini](https://ai.google.dev/models/gemini) is a family of generative AI models that lets developers generate content and solve problems. These models are designed and trained to handle both text and images as input.\n", - "\n", - "[LangChain](https://www.langchain.com/) is a data framework designed to make integration of Large Language Models (LLM) like Gemini easier for applications.\n", - "\n", - "[Chroma](https://docs.trychroma.com/) is an open-source embedding database focused on simplicity and developer productivity. Chroma allows users to store embeddings and their metadata, embed documents and queries, and search the embeddings quickly.\n", - "\n", - "In this notebook, you'll learn how to create an application that answers questions using data from a website with the help of Gemini, LangChain, and Chroma." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "_qRjVe1tZhsx" - }, - "source": [ - "## Setup\n", - "\n", - "First, you must install the packages and set the necessary environment variables.\n", - "\n", - "### Installation\n", - "\n", - "Install LangChain's Python library, `langchain`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "olK4Ejjzuj76" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m798.0/798.0 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.5/1.5 MB\u001b[0m \u001b[31m11.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m216.5/216.5 kB\u001b[0m \u001b[31m10.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m48.4/48.4 kB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m4.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[?25h" - ] - } - ], - "source": [ - "!pip install --quiet langchain" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "G3Ap03KFZjI-" - }, - "source": [ - "Install LangChain's integration package for Gemini, `langchain-google-genai`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "K1CzIZiaurWv" - }, - "outputs": [], - "source": [ - "!pip install --quiet langchain-google-genai" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Ba1VjUO3ZwfS" - }, - "source": [ - "Install Chroma's Python client SDK, `chromadb`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "LGBoQhoz3kdy" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m509.0/509.0 kB\u001b[0m \u001b[31m5.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.4/2.4 MB\u001b[0m \u001b[31m14.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.0/92.0 kB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m60.3/60.3 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m40.6/40.6 kB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.4/5.4 MB\u001b[0m \u001b[31m26.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.4/6.4 MB\u001b[0m \u001b[31m37.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.9/57.9 kB\u001b[0m \u001b[31m5.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m105.6/105.6 kB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.3/67.3 kB\u001b[0m \u001b[31m8.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[?25h Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", - " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", - " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m698.9/698.9 kB\u001b[0m \u001b[31m40.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m51.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.6/67.6 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m70.0/70.0 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m5.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.8/50.8 kB\u001b[0m \u001b[31m6.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m6.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m341.4/341.4 kB\u001b[0m \u001b[31m26.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.4/3.4 MB\u001b[0m \u001b[31m51.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m43.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m130.2/130.2 kB\u001b[0m \u001b[31m14.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m10.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[?25h Building wheel for pypika (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", - "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", - "lida 0.0.10 requires kaleido, which is not installed.\n", - "lida 0.0.10 requires python-multipart, which is not installed.\n", - "tensorflow-probability 0.22.0 requires typing-extensions<4.6.0, but you have typing-extensions 4.9.0 which is incompatible.\u001b[0m\u001b[31m\n", - "\u001b[0m" - ] - } - ], - "source": [ - "!pip install --quiet chromadb" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "wiGHSFmZaniK" - }, - "source": [ - "### Grab an API Key\n", - "\n", - "To use Gemini you need an *API key*. You can create an API key with one click in [Google AI Studio](https://makersuite.google.com/).\n", - "After creating the API key, you can either set an environment variable named `GOOGLE_API_KEY` to your API Key or pass the API key as an argument when using the `ChatGoogleGenerativeAI` class to access Google's `gemini` and `gemini-vision` models or the `GoogleGenerativeAIEmbeddings` class to access Google's Generative AI embedding model using `LangChain`.\n", - "\n", - "In this tutorial, you will set the environment variable `GOOGLE_API_KEY` to configure Gemini to use your API key." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "xId4sR52utS0" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Gemini API Key:··········\n" - ] - } - ], - "source": [ - "# Run this cell and paste the API key in the prompt\n", - "import os\n", - "import getpass\n", - "\n", - "os.environ['GOOGLE_API_KEY'] = getpass.getpass('Gemini API Key:')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "aEKMUyVmckWI" - }, - "source": [ - "## Basic steps\n", - "LLMs are trained offline on a large corpus of public data. Hence they cannot answer questions based on custom or private data accurately without additional context.\n", - "\n", - "If you want to make use of LLMs to answer questions based on private data, you have to provide the relevant documents as context alongside your prompt. This approach is called Retrieval Augmented Generation (RAG).\n", - "\n", - "You will use this approach to create a question-answering assistant using the Gemini text model integrated through LangChain. The assistant is expected to answer questions about the Gemini model. To make this possible you will add more context to the assistant using data from a website.\n", - "\n", - "In this tutorial, you'll implement the two main components in an RAG-based architecture:\n", - "\n", - "1. Retriever\n", - "\n", - " Based on the user's query, the retriever retrieves relevant snippets that add context from the document. In this tutorial, the document is the website data.\n", - " The relevant snippets are passed as context to the next stage - \"Generator\".\n", - "\n", - "2. Generator\n", - "\n", - " The relevant snippets from the website data are passed to the LLM along with the user's query to generate accurate answers.\n", - "\n", - "You'll learn more about these stages in the upcoming sections while implementing the application." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "kPhs4mDkjdgY" - }, - "source": [ - "## Import the required libraries" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "TcvGPVdXu05F" - }, - "outputs": [], - "source": [ - "from langchain import PromptTemplate\n", - "from langchain import hub\n", - "from langchain.docstore.document import Document\n", - "from langchain.document_loaders import WebBaseLoader\n", - "from langchain.schema import StrOutputParser\n", - "from langchain.schema.prompt_template import format_document\n", - "from langchain.schema.runnable import RunnablePassthrough\n", - "from langchain.vectorstores import Chroma" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "4461Jihk_rWq" - }, - "source": [ - "## Retriever\n", - "\n", - "In this stage, you will perform the following steps:\n", - "\n", - "1. Read and parse the website data using LangChain.\n", - "\n", - "2. Create embeddings of the website data.\n", - "\n", - " Embeddings are numerical representations (vectors) of text. Hence, text with similar meaning will have similar embedding vectors. You'll make use of Gemini's embedding model to create the embedding vectors of the website data.\n", - "\n", - "3. Store the embeddings in Chroma's vector store.\n", - " \n", - " Chroma is a vector database. The Chroma vector store helps in the efficient retrieval of similar vectors. Thus, for adding context to the prompt for the LLM, relevant embeddings of the text matching the user's question can be retrieved easily using Chroma.\n", - "\n", - "4. Create a Retriever from the Chroma vector store.\n", - "\n", - " The retriever will be used to pass relevant website embeddings to the LLM along with user queries." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "WomGvIAVjZeI" - }, - "source": [ - "### Read and parse the website data\n", - "\n", - "LangChain provides a wide variety of document loaders. To read the website data as a document, you will use the `WebBaseLoader` from LangChain.\n", - "\n", - "To know more about how to read and parse input data from different sources using the document loaders of LangChain, read LangChain's [document loaders guide](https://python.langchain.com/docs/integrations/document_loaders)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "DeNX9QFM0V-C" - }, - "outputs": [], - "source": [ - "loader = WebBaseLoader(\"https://blog.google/technology/ai/google-gemini-ai/\")\n", - "docs = loader.load()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "olIlIOYrJTlF" - }, - "source": [ - "If you only want to select a specific portion of the website data to add context to the prompt, you can use regex, text slicing, or text splitting.\n", - "\n", - "In this example, you'll use Python's `split()` function to extract the required portion of the text. The extracted text should be converted back to LangChain's `Document` format." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "EDL9YLRb9Bw2" - }, - "outputs": [], - "source": [ - "# Extract the text from the website data document\n", - "text_content = docs[0].page_content\n", - "\n", - "# The text content between the substrings \"code, audio, image and video.\" to\n", - "# \"Cloud TPU v5p\" is relevant for this tutorial. You can use Python's `split()`\n", - "# to select the required content.\n", - "text_content_1 = text_content.split(\"code, audio, image and video.\",1)[1]\n", - "final_text = text_content_1.split(\"Cloud TPU v5p\",1)[0]\n", - "\n", - "# Convert the text to LangChain's `Document` format\n", - "docs = [Document(page_content=final_text, metadata={\"source\": \"local\"})]" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "yDsdAg4Fjo5o" - }, - "source": [ - "### Initialize Gemini's embedding model\n", - "\n", - "To create the embeddings from the website data, you'll use Gemini's embedding model, **embedding-001** which supports creating text embeddings.\n", - "\n", - "To use this embedding model, you have to import `GoogleGenerativeAIEmbeddings` from LangChain. To know more about the embedding model, read Google AI's [language documentation](https://ai.google.dev/models/gemini)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "8NXNTrjp0jdh" - }, - "outputs": [], - "source": [ - "from langchain_google_genai import GoogleGenerativeAIEmbeddings\n", - "\n", - "# If there is no environment variable set for the API key, you can pass the API\n", - "# key to the parameter `google_api_key` of the `GoogleGenerativeAIEmbeddings`\n", - "# function: `google_api_key = \"key\"`.\n", - "\n", - "gemini_embeddings = GoogleGenerativeAIEmbeddings(model=\"models/embedding-001\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "m9Vzw30wpebs" - }, - "source": [ - "### Store the data using Chroma\n", - "\n", - "To create a Chroma vector database from the website data, you will use the `from_documents` function of `Chroma`. Under the hood, this function creates embeddings from the documents created by the document loader of LangChain using any specified embedding model and stores them in a Chroma vector database. \n", - "\n", - "You have to specify the `docs` you created from the website data using LangChain's `WebBasedLoader` and the `gemini_embeddings` as the embedding model when invoking the `from_documents` function to create the vector database from the website data. You can also specify a directory in the `persist_directory` argument to store the vector store on the disk. If you don't specify a directory, the data will be ephemeral in-memory.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "n1VwhUQMvpcN" - }, - "outputs": [], - "source": [ - "# Save to disk\n", - "vectorstore = Chroma.from_documents(\n", - " documents=docs, # Data\n", - " embedding=gemini_embeddings, # Embedding model\n", - " persist_directory=\"./chroma_db\" # Directory to save data\n", - " )" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "WFKyb3JXOeaQ" - }, - "source": [ - "### Create a retriever using Chroma\n", - "\n", - "You'll now create a retriever that can retrieve website data embeddings from the newly created Chroma vector store. This retriever can be later used to pass embeddings that provide more context to the LLM for answering user's queries.\n", - "\n", - "\n", - "To load the vector store that you previously stored in the disk, you can specify the name of the directory that contains the vector store in `persist_directory` and the embedding model in the `embedding_function` arguments of Chroma's initializer.\n", - "\n", - "You can then invoke the `as_retriever` function of `Chroma` on the vector store to create a retriever." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "s3t4kmzIOZQq" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "1\n" - ] - } - ], - "source": [ - "# Load from disk\n", - "vectorstore_disk = Chroma(\n", - " persist_directory=\"./chroma_db\", # Directory of db\n", - " embedding_function=gemini_embeddings # Embedding model\n", - " )\n", - "# Get the Retriever interface for the store to use later.\n", - "# When an unstructured query is given to a retriever it will return documents.\n", - "# Read more about retrievers in the following link.\n", - "# https://python.langchain.com/docs/modules/data_connection/retrievers/\n", - "#\n", - "# Since only 1 document is stored in the Chroma vector store, search_kwargs `k`\n", - "# is set to 1 to decrease the `k` value of chroma's similarity search from 4 to\n", - "# 1. If you don't pass this value, you will get a warning.\n", - "retriever = vectorstore_disk.as_retriever(search_kwargs={\"k\": 1})\n", - "\n", - "# Check if the retriever is working by trying to fetch the relevant docs related\n", - "# to the word 'MMLU' (Massive Multitask Language Understanding). If the length is greater than zero, it means that\n", - "# the retriever is functioning well.\n", - "print(len(retriever.get_relevant_documents(\"MMLU\")))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "LZwcZyRxSO0q" - }, - "source": [ - "## Generator\n", - "\n", - "The Generator prompts the LLM for an answer when the user asks a question. The retriever you created in the previous stage from the Chroma vector store will be used to pass relevant embeddings from the website data to the LLM to provide more context to the user's query.\n", - "\n", - "You'll perform the following steps in this stage:\n", - "\n", - "1. Chain together the following:\n", - " * A prompt for extracting the relevant embeddings using the retriever.\n", - " * A prompt for answering any question using LangChain.\n", - " * An LLM model from Gemini for prompting.\n", - " \n", - "2. Run the created chain with a question as input to prompt the model for an answer.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "FtUi5FBIJMDy" - }, - "source": [ - "### Initialize Gemini\n", - "\n", - "You must import `ChatGoogleGenerativeAI` from LangChain to initialize your model.\n", - " In this example, you will use **gemini-pro**, as it supports text summarization. To know more about the text model, read Google AI's [language documentation](https://ai.google.dev/models/gemini).\n", - "\n", - "You can configure the model parameters such as ***temperature*** or ***top_p***, by passing the appropriate values when initializing the `ChatGoogleGenerativeAI` LLM. To learn more about the parameters and their uses, read Google AI's [concepts guide](https://ai.google.dev/docs/concepts#model_parameters)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "CaA1vRCh7s36" - }, - "outputs": [], - "source": [ - "from langchain_google_genai import ChatGoogleGenerativeAI\n", - "\n", - "# If there is no environment variable set for the API key, you can pass the API\n", - "# key to the parameter `google_api_key` of the `ChatGoogleGenerativeAI` function:\n", - "# `google_api_key=\"key\"`.\n", - "llm = ChatGoogleGenerativeAI(model=\"gemini-pro\",\n", - " temperature=0.7, top_p=0.85)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "jC4QDhiPpDJa" - }, - "source": [ - "### Create prompt templates\n", - "\n", - "You'll use LangChain's [PromptTemplate](https://python.langchain.com/docs/modules/model_io/prompts/prompt_templates/) to generate prompts to the LLM for answering questions.\n", - "\n", - "In the `llm_prompt`, the variable `question` will be replaced later by the input question, and the variable `context` will be replaced by the relevant text from the website retrieved from the Chroma vector store." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "90Czqh074dEC" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "input_variables=['context', 'question'] template=\"You are an assistant for question-answering tasks.\\nUse the following context to answer the question.\\nIf you don't know the answer, just say that you don't know.\\nUse five sentences maximum and keep the answer concise.\\n\\nQuestion: {question} \\nContext: {context} \\nAnswer:\"\n" - ] - } - ], - "source": [ - "# Prompt template to query Gemini\n", - "llm_prompt_template = \"\"\"You are an assistant for question-answering tasks.\n", - "Use the following context to answer the question.\n", - "If you don't know the answer, just say that you don't know.\n", - "Use five sentences maximum and keep the answer concise.\\n\n", - "Question: {question} \\nContext: {context} \\nAnswer:\"\"\"\n", - "\n", - "llm_prompt = PromptTemplate.from_template(llm_prompt_template)\n", - "\n", - "print(llm_prompt)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "KXDh2jsdp4sr" - }, - "source": [ - "### Create a stuff documents chain\n", - "\n", - "LangChain provides [Chains](https://python.langchain.com/docs/modules/chains/) for chaining together LLMs with each other or other components for complex applications. You will create a **stuff documents chain** for this application. A stuff documents chain lets you combine all the relevant documents, insert them into the prompt, and pass that prompt to the LLM.\n", - "\n", - "You can create a stuff documents chain using the [LangChain Expression Language (LCEL)](https://python.langchain.com/docs/expression_language).\n", - "\n", - "To learn more about different types of document chains, read LangChain's [chains guide](https://python.langchain.com/docs/modules/chains/document/).\n", - "\n", - "The stuff documents chain for this application retrieves the relevant website data and passes it as the context to an LLM prompt along with the input question." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "gj5sWzpwp7vc" - }, - "outputs": [], - "source": [ - "# Combine data from documents to readable string format.\n", - "def format_docs(docs):\n", - " return \"\\n\\n\".join(doc.page_content for doc in docs)\n", - "\n", - "# Create stuff documents chain using LCEL.\n", - "#\n", - "# This is called a chain because you are chaining together different elements\n", - "# with the LLM. In the following example, to create the stuff chain, you will\n", - "# combine the relevant context from the website data matching the question, the\n", - "# LLM model, and the output parser together like a chain using LCEL.\n", - "#\n", - "# The chain implements the following pipeline:\n", - "# 1. Extract the website data relevant to the question from the Chroma\n", - "# vector store and save it to the variable `context`.\n", - "# 2. `RunnablePassthrough` option to provide `question` when invoking\n", - "# the chain.\n", - "# 3. The `context` and `question` are then passed to the prompt where they\n", - "# are populated in the respective variables.\n", - "# 4. This prompt is then passed to the LLM (`gemini-pro`).\n", - "# 5. Output from the LLM is passed through an output parser\n", - "# to structure the model's response.\n", - "rag_chain = (\n", - " {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n", - " | llm_prompt\n", - " | llm\n", - " | StrOutputParser()\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "cPPqsGCLIrs1" - }, - "source": [ - "### Prompt the model\n", - "\n", - "You can now query the LLM by passing any question to the `invoke()` function of the stuff documents chain you created previously." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "4vIaopCsIq0B" - }, - "outputs": [ - { - "data": { - "application/vnd.google.colaboratory.intrinsic+json": { - "type": "string" - }, - "text/plain": [ - "\"Gemini is Google's largest and most capable AI model, designed to efficiently run on various platforms, from data centers to mobile devices. It excels in understanding and reasoning about text, images, audio, and code. Gemini's sophisticated multimodal reasoning capabilities enable it to uncover knowledge from vast amounts of data and explain reasoning in complex subjects like math and physics. It can also generate high-quality code in multiple programming languages.\"" - ] - }, - "execution_count": 15, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "rag_chain.invoke(\"What is Gemini?\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "lV7T9rqDdjZK" - }, - "source": [ - "# Conclusion\n", - "\n", - "That's it. You have successfully created an LLM application that answers questions using data from a website with the help of Gemini, LangChain, and Chroma." - ] - } - ], - "metadata": { - "colab": { - "name": "Gemini_LangChain_QA_Chroma_WebLoad.ipynb", - "toc_visible": true - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} diff --git a/examples/gemini/python/langchain/Gemini_LangChain_QA_Pinecone_WebLoad.ipynb b/examples/gemini/python/langchain/Gemini_LangChain_QA_Pinecone_WebLoad.ipynb deleted file mode 100644 index 6090249d6..000000000 --- a/examples/gemini/python/langchain/Gemini_LangChain_QA_Pinecone_WebLoad.ipynb +++ /dev/null @@ -1,752 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "YdsMOBaBfyT0" - }, - "source": [ - "##### Copyright 2024 Google LLC." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "rIIf_RgOf3sr" - }, - "outputs": [], - "source": [ - "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", - "# you may not use this file except in compliance with the License.\n", - "# You may obtain a copy of the License at\n", - "#\n", - "# https://www.apache.org/licenses/LICENSE-2.0\n", - "#\n", - "# Unless required by applicable law or agreed to in writing, software\n", - "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", - "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", - "# See the License for the specific language governing permissions and\n", - "# limitations under the License." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "TySweisNf_Am" - }, - "source": [ - "# Question Answering using Gemini, LangChain, and Pinecone" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "awKO767lQIWh" - }, - "source": [ - "\n", - " \n", - " \n", - "
\n", - " Run in Google Colab\n", - " \n", - " View source on GitHub\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "bA5Hys5PU_nt" - }, - "source": [ - "## Overview\n", - "\n", - "[Gemini](https://ai.google.dev/models/gemini) is a family of generative AI models that lets developers generate content and solve problems. These models are designed and trained to handle both text and images as input.\n", - "\n", - "[LangChain](https://www.langchain.com/) is a data framework designed to make integration of Large Language Models (LLM) like Gemini easier for applications.\n", - "\n", - "[Pinecone](https://www.pinecone.io/) is a cloud-first vector database that allows users to search across billions of embeddings with ultra-low query latency.\n", - "\n", - "In this notebook, you'll learn how to create an application that answers questions using data from a website with the help of Gemini, LangChain, and Pinecone." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "_qRjVe1tZhsx" - }, - "source": [ - "## Setup\n", - "\n", - "First, you must install the packages and set the necessary environment variables.\n", - "\n", - "### Installation\n", - "\n", - "Install `LangChain`'s python library." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "olK4Ejjzuj76" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m817.0/817.0 kB\u001b[0m \u001b[31m14.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m62.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m250.8/250.8 kB\u001b[0m \u001b[31m28.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m63.1/63.1 kB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m6.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.5/138.5 kB\u001b[0m \u001b[31m16.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[?25h" - ] - } - ], - "source": [ - "!pip install --quiet langchain" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "G3Ap03KFZjI-" - }, - "source": [ - "Install LangChain's integration package for Gemini, `langchain-google-genai`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "K1CzIZiaurWv" - }, - "outputs": [], - "source": [ - "!pip install --quiet langchain-google-genai" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "V7Y51x2AexEf" - }, - "source": [ - "Install LangChain's integration package for the new version of Pinecone, `langchain-pinecone`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "kSxJt9NCerJX" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/211.0 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m\u001b[90m━━━\u001b[0m \u001b[32m194.6/211.0 kB\u001b[0m \u001b[31m5.7 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.0/211.0 kB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[?25h" - ] - } - ], - "source": [ - "!pip install --quiet langchain-pinecone" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Ba1VjUO3ZwfS" - }, - "source": [ - "Install Pinecone's python client SDK, `pinecone-client`" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "LGBoQhoz3kdy" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/201.4 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[91m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[91m╸\u001b[0m\u001b[90m━\u001b[0m \u001b[32m194.6/201.4 kB\u001b[0m \u001b[31m5.7 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m201.4/201.4 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[?25h" - ] - } - ], - "source": [ - "!pip install --quiet pinecone-client==3.0.2" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "myebeBlLkqVN" - }, - "source": [ - "### Grab a Gemini API Key\n", - "\n", - "To use Gemini you need an *API key*. You can create an API key with one click in [Google AI Studio](https://makersuite.google.com/).\n", - "After creating the API key, you can either set an environment variable named `GOOGLE_API_KEY` to your API Key or pass the API key as an argument when using the `ChatGoogleGenerativeAI` class to access Google's `gemini-1.5-flash` or `gemini-1.5-pro` models or the `GoogleGenerativeAIEmbeddings` class to access Google's Generative AI embedding model using `LangChain`.\n", - "\n", - "In this tutorial, you will set the environment variable `GOOGLE_API_KEY` to configure Gemini to use your API key." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "xId4sR52utS0" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Gemini API Key:··········\n" - ] - } - ], - "source": [ - "# Run this cell and paste the API key in the prompt\n", - "import os\n", - "import getpass\n", - "\n", - "os.environ['GOOGLE_API_KEY'] = getpass.getpass('Gemini API Key:')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "MPQLjFvRooqn" - }, - "source": [ - "### Setup Pinecone\n", - "\n", - "To use Pinecone in your application, you must have an API key. To create an API key you have to set up a Pinecone account. Visit [Pinecone's app page](https://app.pinecone.io/), and Sign up/Log in to your account. Then navigate to the \"API Keys\" section and copy your API key.\n", - "\n", - "For more detailed instructions on getting the API key, you can read Pinecone's [Quickstart documentation](https://docs.pinecone.io/docs/quickstart#2-get-your-api-key).\n", - "\n", - "Set the environment variable `PINECONE_API_KEY` to configure Pinecone to use your API key.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "A7jTZLEApgtm" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Pinecone API Key:··········\n" - ] - } - ], - "source": [ - "os.environ['PINECONE_API_KEY'] = getpass.getpass('Pinecone API Key:')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "YGOKV3XflBCe" - }, - "source": [ - "## Basic steps\n", - "LLMs are trained offline on a large corpus of public data. Hence they cannot answer questions based on custom or private data accurately without additional context.\n", - "\n", - "If you want to make use of LLMs to answer questions based on private data, you have to provide the relevant documents as context alongside your prompt. This approach is called Retrieval Augmented Generation (RAG).\n", - "\n", - "You will use this approach to create a question-answering assistant using the Gemini text model integrated through LangChain. The assistant is expected to answer questions about Gemini model. To make this possible you will add more context to the assistant using data from a website.\n", - "\n", - "In this tutorial, you'll implement the two main components in an RAG-based architecture:\n", - "\n", - "1. Retriever\n", - "\n", - " Based on the user's query, the retriever retrieves relevant snippets that add context from the document. In this tutorial, the document is the website data.\n", - " The relevant snippets are passed as context to the next stage - \"Generator\".\n", - "\n", - "2. Generator\n", - "\n", - " The relevant snippets from the website data are passed to the LLM along with the user's query to generate accurate answers.\n", - "\n", - "You'll learn more about these stages in the upcoming sections while implementing the application." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "kPhs4mDkjdgY" - }, - "source": [ - "## Import the required libraries" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "TcvGPVdXu05F" - }, - "outputs": [], - "source": [ - "from langchain import hub\n", - "from langchain import PromptTemplate\n", - "from langchain.docstore.document import Document\n", - "from langchain.document_loaders import WebBaseLoader\n", - "from langchain.schema import StrOutputParser\n", - "from langchain.schema.prompt_template import format_document\n", - "from langchain.schema.runnable import RunnablePassthrough\n", - "from langchain.text_splitter import RecursiveCharacterTextSplitter\n", - "from langchain_pinecone import Pinecone\n", - "\n", - "from pinecone import Pinecone as pc\n", - "from pinecone import PodSpec" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "qZ3tM0T2lbVm" - }, - "source": [ - "## Retriever\n", - "\n", - "In this stage, you will perform the following steps:\n", - "\n", - "1. Read and parse the website data using LangChain.\n", - "\n", - "2. Create embeddings of the website data.\n", - "\n", - " Embeddings are numerical representations (vectors) of text. Hence, text with similar meaning will have similar embedding vectors. You'll make use of Gemini's embedding model to create the embedding vectors of the website data.\n", - "\n", - "3. Store the embeddings in Pinecone's vector store.\n", - " \n", - " Pinecone is a vector database. The Pinecone vector store helps in the efficient retrieval of similar vectors. Thus, for adding context to the prompt for the LLM, relevant embeddings of the text matching the user's question can be retrieved easily using Pinecone.\n", - "\n", - "4. Create a Retriever from the Pinecone vector store.\n", - "\n", - " The retriever will be used to pass relevant website embeddings to the LLM along with user queries." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "W2N-NCPElqN3" - }, - "source": [ - "### Read and parse the website data\n", - "\n", - "LangChain provides a wide variety of document loaders. To read the website data as a document, you will use the `WebBaseLoader` from LangChain.\n", - "\n", - "To know more about how to read and parse input data from different sources using the document loaders of LangChain, read LangChain's [document loaders guide](https://python.langchain.com/docs/integrations/document_loaders)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "DeNX9QFM0V-C" - }, - "outputs": [], - "source": [ - "loader = WebBaseLoader(\"https://blog.google/technology/ai/google-gemini-ai/\")\n", - "docs = loader.load()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "y2N6RoTDlwsM" - }, - "source": [ - "If you only want to select a specific portion of the website data to add context to the prompt, you can use regex, text slicing, or text splitting.\n", - "\n", - "In this example, you'll use Python's `split()` function to extract the required portion of the text. The extracted text should be converted back to LangChain's `Document` format." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "qOwDregSBVVG" - }, - "outputs": [], - "source": [ - "# Extract the text from the website data document\n", - "text_content = docs[0].page_content\n", - "# The text content between the substrings \"code, audio, image and video.\" to\n", - "# \"Cloud TPU v5p\" is relevant for this tutorial. You can use Python's `split()`\n", - "# to select the required content.\n", - "text_content_1 = text_content.split(\"code, audio, image and video.\",1)[1]\n", - "final_text = text_content_1.split(\"Cloud TPU v5p\",1)[0]\n", - "\n", - "# Convert the text to LangChain's `Document` format\n", - "docs = [Document(page_content=final_text, metadata={\"source\": \"local\"})]" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "sgGVAFqWl20v" - }, - "source": [ - "### Initialize Gemini's embedding model\n", - "\n", - "To create the embeddings from the website data, you'll use Gemini's embedding model, **embedding-001** which supports creating text embeddings.\n", - "\n", - "To use this embedding model, you have to import `GoogleGenerativeAIEmbeddings` from LangChain. To know more about the embedding model, read Google AI's [language documentation](https://ai.google.dev/models/gemini)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "8NXNTrjp0jdh" - }, - "outputs": [], - "source": [ - "from langchain_google_genai import GoogleGenerativeAIEmbeddings\n", - "\n", - "# If there is no environment variable set for the API key, you can pass the API\n", - "# key to the parameter `google_api_key` of the `GoogleGenerativeAIEmbeddings`\n", - "# function: `google_api_key = \"key\"`.\n", - "\n", - "gemini_embeddings = GoogleGenerativeAIEmbeddings(model=\"models/embedding-001\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Zr5xeWUXmnUe" - }, - "source": [ - "### Store the data using Pinecone\n", - "\n", - "\n", - "To create a Pinecone vector database, first, you have to initialize your Pinecone client connection using the API key you set previously.\n", - "\n", - "In Pinecone, vector embeddings have to be stored in indexes. An index represents the vector data's top-level organizational unit. The vectors in any index must have the same dimensionality and distance metric for calculating similarity. You can read more about indexes in [Pinecone's Indexes documentation](https://docs.pinecone.io/docs/indexes).\n", - "\n", - "First, you'll create an index using Pinecone's `create_index` function. Pinecone allows you to create two types of indexes, Serverless indexes and Pod-based indexes. Pinecone's free starter plan lets you create only one project and one pod-based starter index with sufficient resources to support 100,000 vectors. For this tutorial, you have to create a pod-based starter index. To know more about different indexes and how they can be created, read Pinecone's [create indexes guide](https://docs.pinecone.io/docs/new-api#creating-indexes).\n", - "\n", - "\n", - "Next, you'll insert the documents you extracted earlier from the website data into the newly created index using LangChain's `Pinecone.from_documents`. Under the hood, this function creates embeddings from the documents created by the document loader of LangChain using any specified embedding model and inserts them into the specified index in a Pinecone vector database. \n", - "\n", - "You have to specify the `docs` you created from the website data using LangChain's `WebBasedLoader` and the `gemini_embeddings` as the embedding model when invoking the `from_documents` function to create the vector database from the website data." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "n1VwhUQMvpcN" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "creating index\n", - "{'dimension': 768,\n", - " 'host': 'langchain-demo-xcfkw2a.svc.gcp-starter.pinecone.io',\n", - " 'metric': 'cosine',\n", - " 'name': 'langchain-demo',\n", - " 'spec': {'pod': {'environment': 'gcp-starter',\n", - " 'pod_type': 'starter',\n", - " 'pods': 1,\n", - " 'replicas': 1,\n", - " 'shards': 1}},\n", - " 'status': {'ready': True, 'state': 'Ready'}}\n" - ] - } - ], - "source": [ - "# Initialize Pinecone client\n", - "\n", - "pine_client= pc(\n", - " api_key = os.getenv(\"PINECONE_API_KEY\"), # API key from app.pinecone.io\n", - " )\n", - "index_name = \"langchain-demo\"\n", - "\n", - "# First, check if the index already exists. If it doesn't, create a new one.\n", - "if index_name not in pine_client.list_indexes().names():\n", - " # Create a new index.\n", - " # https://docs.pinecone.io/docs/new-api#creating-a-starter-index\n", - " print(\"Creating index\")\n", - " pine_client.create_index(name=index_name,\n", - " # `cosine` distance metric compares different documents\n", - " # for similarity.\n", - " # Read more about different distance metrics from\n", - " # https://docs.pinecone.io/docs/indexes#distance-metrics.\n", - " metric=\"cosine\",\n", - " # The Gemini embedding model `embedding-001` uses\n", - " # 768 dimensions.\n", - " dimension=768,\n", - " # Specify the pod details.\n", - " spec=PodSpec(\n", - " # Starter indexes are hosted in the `gcp-starter`\n", - " # environment.\n", - " environment=\"gcp-starter\",\n", - " pod_type=\"starter\",\n", - " pods=1)\n", - " )\n", - " print(pine_client.describe_index(index_name))\n", - "\n", - "# If there is no environment variable set for the API key, you can pass the API\n", - "# key to the parameter `pinecone_api_key` of the `Pinecone.from_documents`\n", - "# function: `pinecone_api_key = \"key\"`.\n", - "vectorstore = Pinecone.from_documents(docs,\n", - " gemini_embeddings, index_name=index_name)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "BuSjapvHnc6T" - }, - "source": [ - "### Create a retriever using Pinecone\n", - "\n", - "You'll now create a retriever that can retrieve website data embeddings from the newly created Pinecone vector store. This retriever can be later used to pass embeddings that provide more context to the LLM for answering user's queries.\n", - "\n", - "Invoke the `as_retriever` function of the vector store you initialized in the last step, to create a retriever." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "qndTwf0tnQDv" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "0\n" - ] - } - ], - "source": [ - "retriever = vectorstore.as_retriever()\n", - "# Check if the retriever is working by trying to fetch the relevant docs related\n", - "# to the word 'MMLU'(Massive Multitask Language Understanding). If the length is\n", - "# greater than zero, it means that the retriever is functioning well.\n", - "print(len(retriever.get_relevant_documents(\"MMLU\")))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "7Qw00lvPnjfR" - }, - "source": [ - "## Generator\n", - "\n", - "The Generator prompts the LLM for an answer when the user asks a question. The retriever you created in the previous stage from the Pinecone vector store will be used to pass relevant embeddings from the website data to the LLM to provide more context to the user's query.\n", - "\n", - "You'll perform the following steps in this stage:\n", - "\n", - "1. Chain together the following:\n", - " * A prompt for extracting the relevant embeddings using the retriever.\n", - " * A prompt for answering any question using LangChain.\n", - " * An LLM model from Gemini for prompting.\n", - " \n", - "2. Run the created chain with a question as input to prompt the model for an answer.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "c2MK2wLwnkLg" - }, - "source": [ - "### Initialize Gemini\n", - "\n", - "You must import `ChatGoogleGenerativeAI` from LangChain to initialize your model.\n", - " In this example, you will use **gemini-pro**, as it supports text summarization. To know more about the text model, read Google AI's [language documentation](https://ai.google.dev/models/gemini).\n", - "\n", - "You can configure the model parameters such as ***temperature*** or ***top_p***, by passing the appropriate values when initializing the `ChatGoogleGenerativeAI` LLM. To learn more about the parameters and their uses, read Google AI's [concepts guide](https://ai.google.dev/docs/concepts#model_parameters)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "CaA1vRCh7s36" - }, - "outputs": [], - "source": [ - "from langchain_google_genai import ChatGoogleGenerativeAI\n", - "\n", - "# If there is no environment variable set for the API key, you can pass the API\n", - "# key to the parameter `google_api_key` of the `ChatGoogleGenerativeAI`\n", - "# function: `google_api_key=\"key\"`.\n", - "\n", - "llm = ChatGoogleGenerativeAI(model=\"gemini-pro\",\n", - " temperature=0.7, top_p=0.85)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "2BeLN6RXnuS2" - }, - "source": [ - "### Create prompt templates\n", - "\n", - "You'll use LangChain's [PromptTemplate](https://python.langchain.com/docs/modules/model_io/prompts/prompt_templates/) to generate prompts to the LLM for answering questions.\n", - "\n", - "In the `llm_prompt`, the variable `question` will be replaced later by the input question, and the variable `context` will be replaced by the relevant text from the website retrieved from the Pinecone vector store." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "90Czqh074dEC" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "input_variables=['context', 'question'] template=\"You are an assistant for question-answering tasks.\\nUse the following context to answer the question.\\nIf you don't know the answer, just say that you don't know.\\nUse five sentences maximum and keep the answer concise.\\n\\nQuestion: {question} \\nContext: {context} \\nAnswer:\"\n" - ] - } - ], - "source": [ - "# Prompt template to query Gemini\n", - "llm_prompt_template = \"\"\"You are an assistant for question-answering tasks.\n", - "Use the following context to answer the question.\n", - "If you don't know the answer, just say that you don't know.\n", - "Use five sentences maximum and keep the answer concise.\n", - "\n", - "Question: {question}\n", - "Context: {context}\n", - "Answer:\"\"\"\n", - "\n", - "llm_prompt = PromptTemplate.from_template(llm_prompt_template)\n", - "\n", - "print(llm_prompt)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "TkWpzMmpnx7b" - }, - "source": [ - "### Create a stuff documents chain\n", - "\n", - "LangChain provides [Chains](https://python.langchain.com/docs/modules/chains/) for chaining together LLMs with each other or other components for complex applications. You will create a **stuff documents chain** for this application. A stuff documents chain lets you combine all the relevant documents, insert them into the prompt, and pass that prompt to the LLM.\n", - "\n", - "You can create a stuff documents chain using the [LangChain Expression Language (LCEL)](https://python.langchain.com/docs/expression_language).\n", - "\n", - "To learn more about different types of document chains, read LangChain's [chains guide](https://python.langchain.com/docs/modules/chains/document/).\n", - "\n", - "The stuff documents chain for this application retrieves the relevant website data and passes it as the context to an LLM prompt along with the input question." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "gj5sWzpwp7vc" - }, - "outputs": [], - "source": [ - "# Combine data from documents to readable string format.\n", - "def format_docs(docs):\n", - " return \"\\n\\n\".join(doc.page_content for doc in docs)\n", - "\n", - "# Create stuff documents chain using LCEL.\n", - "# This is called a chain because you are chaining\n", - "# together different elements with the LLM.\n", - "# In the following example, to create a stuff chain,\n", - "# you will combine content, prompt, LLM model, and\n", - "# output parser together like a chain using LCEL.\n", - "#\n", - "# The chain implements the following pipeline:\n", - "# 1. Extract data from documents and save to the variable `context`.\n", - "# 2. Use the `RunnablePassthrough` option to provide question during invoke.\n", - "# 3. The `context` and `question` are then passed to the prompt and\n", - "# input variables in the prompt are populated.\n", - "# 4. The prompt is then passed to the LLM (`gemini-pro`).\n", - "# 5. Output from the LLM is passed through an output parser\n", - "# to structure the model response.\n", - "rag_chain = (\n", - " {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n", - " | llm_prompt\n", - " | llm\n", - " | StrOutputParser()\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "gmHx_F7DoMgM" - }, - "source": [ - "### Prompt the model\n", - "\n", - "You can now query the LLM by passing any question to the `invoke()` function of the stuff documents chain you created previously." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "95W-sbTjoGGj" - }, - "outputs": [ - { - "data": { - "application/vnd.google.colaboratory.intrinsic+json": { - "type": "string" - }, - "text/plain": [ - "\"Gemini is Google's largest and most capable AI model. It is also their most flexible model, able to efficiently run on everything from data centers to mobile devices. Its state-of-the-art capabilities significantly enhance the way developers and enterprise customers build and scale with AI.\"" - ] - }, - "execution_count": 18, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "rag_chain.invoke(\"What is Gemini?\")" - ] - } - ], - "metadata": { - "colab": { - "name": "Gemini_LangChain_QA_Pinecone_WebLoad.ipynb", - "toc_visible": true - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} diff --git a/examples/gemini/python/langchain/Gemini_LangChain_Summarization_WebLoad.ipynb b/examples/gemini/python/langchain/Gemini_LangChain_Summarization_WebLoad.ipynb deleted file mode 100644 index 3e789669e..000000000 --- a/examples/gemini/python/langchain/Gemini_LangChain_Summarization_WebLoad.ipynb +++ /dev/null @@ -1,435 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "Tce3stUlHN0L" - }, - "source": [ - "##### Copyright 2024 Google LLC." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "tuOe1ymfHZPu" - }, - "outputs": [], - "source": [ - "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", - "# you may not use this file except in compliance with the License.\n", - "# You may obtain a copy of the License at\n", - "#\n", - "# https://www.apache.org/licenses/LICENSE-2.0\n", - "#\n", - "# Unless required by applicable law or agreed to in writing, software\n", - "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", - "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", - "# See the License for the specific language governing permissions and\n", - "# limitations under the License." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "f22a409c18ef" - }, - "source": [ - "# Summarize large documents using LangChain and Gemini" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "awKO767lQIWh" - }, - "source": [ - "\n", - " \n", - " \n", - "
\n", - " Run in Google Colab\n", - " \n", - " View source on GitHub\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "f892e8b2c8ef" - }, - "source": [ - "## Overview\n", - "\n", - "[Gemini](https://ai.google.dev/models/gemini) is a family of generative AI models that lets developers generate content and solve problems. These models are designed and trained to handle both text and images as input.\n", - "\n", - "[LangChain](https://www.langchain.com/) is a framework designed to make integration of Large Language Models (LLM) like Gemini easier for applications.\n", - "\n", - "In this notebook, you'll learn how to create an application to summarize large documents using Gemini and LangChain.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "iHj4T7hsx1EB" - }, - "source": [ - "## Setup\n", - "\n", - "First, you must install the packages and set the necessary environment variables.\n", - "\n", - "### Installation\n", - "\n", - "Install LangChain's Python library, `langchain`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "yERdO0eFJpb-" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m803.1/803.1 kB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.5/1.5 MB\u001b[0m \u001b[31m9.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m205.7/205.7 kB\u001b[0m \u001b[31m9.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.7/46.7 kB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[?25h" - ] - } - ], - "source": [ - "!pip install --quiet langchain" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "45MVc1stzNUN" - }, - "source": [ - "Install LangChain's integration package for Gemini, `langchain-google-genai`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "FcMXJTN5JsfU" - }, - "outputs": [], - "source": [ - "!pip install --quiet langchain-google-genai" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "ycFMUTxn0VoI" - }, - "source": [ - "### Grab an API Key\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "e1dZNWvUzksX" - }, - "source": [ - "To use Gemini you need an *API key*. You can create an API key with one click in [Google AI Studio](https://makersuite.google.com/).\n", - "After creating the API key, you can either set an environment variable named `GOOGLE_API_KEY` to your API Key or pass the API key as an argument when creating the `ChatGoogleGenerativeAI` LLM using `LangChain`.\n", - "\n", - "In this tutorial, you will set the environment variable `GOOGLE_API_KEY` to configure Gemini to use your API key." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "1RO5jvTTddtc" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Gemini API Key:··········\n" - ] - } - ], - "source": [ - "# Run this cell and paste the API key in the prompt\n", - "import os\n", - "import getpass\n", - "\n", - "os.environ['GOOGLE_API_KEY'] = getpass.getpass('Gemini API Key:')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "i7wgsoiz418u" - }, - "source": [ - "## Summarize text\n", - "\n", - "In this tutorial, you are going to summarize the text from a website using the Gemini model integrated through LangChain.\n", - "\n", - "You'll perform the following steps to achieve the same:\n", - "1. Read and parse the website data using LangChain.\n", - "2. Chain together the following:\n", - " * A prompt for extracting the required input data from the parsed website data.\n", - " * A prompt for summarizing the text using LangChain.\n", - " * An LLM model (Gemini) for prompting.\n", - "\n", - "3. Run the created chain to prompt the model for the summary of the website data." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "RhL92-zmSB6Z" - }, - "source": [ - "### Import the required libraries" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "rAv0UicpKARZ" - }, - "outputs": [], - "source": [ - "from langchain import PromptTemplate\n", - "from langchain.document_loaders import WebBaseLoader\n", - "from langchain.schema import StrOutputParser\n", - "from langchain.schema.prompt_template import format_document" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "4tKpRvmMRX23" - }, - "source": [ - "### Read and parse the website data\n", - "\n", - "LangChain provides a wide variety of document loaders. To read the website data as a document, you will use the `WebBaseLoader` from LangChain.\n", - "\n", - "To know more about how to read and parse input data from different sources using the document loaders of LangChain, read LangChain's [document loaders guide](https://python.langchain.com/docs/integrations/document_loaders)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "TTgmyxXzKCSq" - }, - "outputs": [], - "source": [ - "loader = WebBaseLoader(\"https://blog.google/technology/ai/google-gemini-ai/#sundar-note\")\n", - "docs = loader.load()\n", - "\n", - "print(docs)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "4xlf_F_4B6lB" - }, - "source": [ - "### Initialize Gemini\n", - "\n", - "You must import the `ChatGoogleGenerativeAI` LLM from LangChain to initialize your model.\n", - " In this example you will use **gemini-pro**, as it supports text summarization. To know more about the text model, read Google AI's [language documentation](https://ai.google.dev/models/gemini).\n", - "\n", - "You can configure the model parameters such as ***temperature*** or ***top_p***, by passing the appropriate values when creating the `ChatGoogleGenerativeAI` LLM. To learn more about the parameters and their uses, read Google AI's [concepts guide](https://ai.google.dev/docs/concepts#model_parameters)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "WWA9F0ZqB-8k" - }, - "outputs": [], - "source": [ - "from langchain_google_genai import ChatGoogleGenerativeAI\n", - "\n", - "# If there is no env variable set for API key, you can pass the API key\n", - "# to the parameter `google_api_key` of the `ChatGoogleGenerativeAI` function:\n", - "# `google_api_key=\"key\"`.\n", - "\n", - "llm = ChatGoogleGenerativeAI(model=\"gemini-pro\",\n", - " temperature=0.7, top_p=0.85)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "6TECDzaUSTvS" - }, - "source": [ - "### Create prompt templates\n", - "\n", - "You'll use LangChain's [PromptTemplate](https://python.langchain.com/docs/modules/model_io/prompts/prompt_templates/) to generate prompts for summarizing the text.\n", - "\n", - "To summarize the text from the website, you will need the following prompts.\n", - "1. Prompt to extract the data from the output of `WebBaseLoader`, named `doc_prompt`\n", - "2. Prompt for the LLM model (Gemini) to summarize the extracted text, named `llm_prompt`.\n", - "\n", - "In the `llm_prompt`, the variable `text` will be replaced later by the text from the website." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "rixvvvaNKLe_" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "input_variables=['text'] template='Write a concise summary of the following:\\n\"{text}\"\\nCONCISE SUMMARY:'\n" - ] - } - ], - "source": [ - "# To extract data from WebBaseLoader\n", - "doc_prompt = PromptTemplate.from_template(\"{page_content}\")\n", - "\n", - "# To query Gemini\n", - "llm_prompt_template = \"\"\"Write a concise summary of the following:\n", - "\"{text}\"\n", - "CONCISE SUMMARY:\"\"\"\n", - "llm_prompt = PromptTemplate.from_template(llm_prompt_template)\n", - "\n", - "print(llm_prompt)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "-wPBMFyISh13" - }, - "source": [ - "### Create a Stuff documents chain\n", - "\n", - "LangChain provides [Chains](https://python.langchain.com/docs/modules/chains/) for chaining together LLMs with each other or other components for complex applications. You will create a **Stuff documents chain** for this application. A **Stuff documents chain** lets you combine all the documents, insert them into the prompt and pass that prompt to the LLM.\n", - "\n", - "You can create a Stuff documents chain using the [LangChain Expression Language (LCEL)](https://python.langchain.com/docs/expression_language).\n", - "\n", - "To learn more about different types of document chains, read LangChain's [chains guide](https://python.langchain.com/docs/modules/chains/document/)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "EMZomQdyKMr5" - }, - "outputs": [], - "source": [ - "# Create Stuff documents chain using LCEL.\n", - "# This is called a chain because you are chaining\n", - "# together different elements with the LLM.\n", - "# In the following example, to create stuff chain,\n", - "# you will combine content, prompt, LLM model and\n", - "# output parser together like a chain using LCEL.\n", - "#\n", - "# The chain implements the following pipeline:\n", - "# 1. Extract data from documents and save to variable `text`.\n", - "# 2. This `text` is then passed to the prompt and input variable\n", - "# in prompt is populated.\n", - "# 3. The prompt is then passed to the LLM (Gemini).\n", - "# 4. Output from the LLM is passed through an output parser\n", - "# to structure the model response.\n", - "\n", - "stuff_chain = (\n", - " # Extract data from the documents and add to the key `text`.\n", - " {\n", - " \"text\": lambda docs: \"\\n\\n\".join(\n", - " format_document(doc, doc_prompt) for doc in docs\n", - " )\n", - " }\n", - " | llm_prompt # Prompt for Gemini\n", - " | llm # Gemini function\n", - " | StrOutputParser() # output parser\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "5L0Tvk_5eQzC" - }, - "source": [ - "### Prompt the model\n", - "\n", - "To generate the summary of the the website data, pass the documents extracted using the `WebBaseLoader` (`docs`) to `invoke()`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "k9_GxkA5ePRR" - }, - "outputs": [ - { - "data": { - "application/vnd.google.colaboratory.intrinsic+json": { - "type": "string" - }, - "text/plain": [ - "\"Google introduces Gemini, its most capable AI model yet. Gemini is multimodal, flexible, and optimized for different sizes. It surpasses state-of-the-art performance on various benchmarks, including text, coding, and multimodal tasks. Gemini's capabilities include sophisticated reasoning, understanding text, images, audio, and advanced coding. It is designed with responsibility and safety at its core, undergoing comprehensive safety evaluations and incorporating safety classifiers. Gemini is being rolled out across Google products, including Bard, Pixel, Search, and Ads. Developers and enterprise customers can access Gemini Pro via the Gemini API. Gemini Ultra will be available to select partners and experts for early experimentation before a broader release. Gemini represents a new era of AI innovation, with future versions expected to advance planning, memory, and context processing capabilities.\"" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "stuff_chain.invoke(docs)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "nfrBsxUFgZzc" - }, - "source": [ - "# Conclusion\n", - "\n", - "That's it. You have successfully created an LLM application to summarize text using LangChain and Gemini." - ] - } - ], - "metadata": { - "colab": { - "name": "Gemini_LangChain_Summarization_WebLoad.ipynb", - "toc_visible": true - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} diff --git a/examples/gemini/python/llamaindex/Gemini_LlamaIndex_QA_Chroma_WebPageReader.ipynb b/examples/gemini/python/llamaindex/Gemini_LlamaIndex_QA_Chroma_WebPageReader.ipynb deleted file mode 100644 index 7e1dd0250..000000000 --- a/examples/gemini/python/llamaindex/Gemini_LlamaIndex_QA_Chroma_WebPageReader.ipynb +++ /dev/null @@ -1,628 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "MctthPQNUiMt" - }, - "source": [ - "##### Copyright 2024 Google LLC." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "0sK9GK2mUr4Z" - }, - "outputs": [], - "source": [ - "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", - "# you may not use this file except in compliance with the License.\n", - "# You may obtain a copy of the License at\n", - "#\n", - "# https://www.apache.org/licenses/LICENSE-2.0\n", - "#\n", - "# Unless required by applicable law or agreed to in writing, software\n", - "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", - "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", - "# See the License for the specific language governing permissions and\n", - "# limitations under the License." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "EecCBd3afA7C" - }, - "source": [ - "# Question Answering using Gemini, LlamaIndex, and Chroma" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "XhAqH8SXfLhn" - }, - "source": [ - "\n", - " \n", - " \n", - "
\n", - " Run in Google Colab\n", - " \n", - " View source on GitHub\n", - "
" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "itTLwgnvkrfD" - }, - "source": [ - "## Overview\n", - "\n", - "[Gemini](https://ai.google.dev/models/gemini) is a family of generative AI models that lets developers generate content and solve problems. These models are designed and trained to handle both text and images as input.\n", - "\n", - "[LlamaIndex](https://www.llamaindex.ai/) is a simple, flexible data framework that can be used by Large Language Model(LLM) applications to connect custom data sources to LLMs.\n", - "\n", - "[Chroma](https://docs.trychroma.com/) is an open-source embedding database focused on simplicity and developer productivity. Chroma allows users to store embeddings and their metadata, embed documents and queries, and search the embeddings quickly.\n", - "\n", - "In this notebook, you'll learn how to create an application that answers questions using data from a website with the help of Gemini, LlamaIndex, and Chroma." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "fZQLFShAnNDA" - }, - "source": [ - "## Setup\n", - "\n", - "First, you must install the packages and set the necessary environment variables.\n", - "\n", - "### Installation\n", - "\n", - "Install LlamaIndex's python library, `llama-index`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "iK3BOx4u6ssQ" - }, - "outputs": [], - "source": [ - "# This guide was tested with 0.10.17, but feel free to try newer versions.\n", - "!pip install -q llama-index==0.10.17" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "mzd3CN4yKGpw" - }, - "source": [ - "Install LlamaIndex's integration package for Gemini, `llama-index-llms-gemini`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "aUn_Pdp_KGyM" - }, - "outputs": [], - "source": [ - "!pip install -q llama-index-llms-gemini" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "vWqDXJhuKG-B" - }, - "source": [ - "Install LlamaIndex's integration package for Gemini embedding model, `llama-index-embeddings-gemini`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "IdAKIYmdKHH3" - }, - "outputs": [], - "source": [ - "!pip install -q llama-index-embeddings-gemini" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "_9XYs842I-6G" - }, - "source": [ - "Install LlamaIndex's web page reader, `llama-index-readers-web`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "HnFd2OWXI_M-" - }, - "outputs": [], - "source": [ - "!pip install -q llama-index-readers-web" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "muHuC24HoCBq" - }, - "source": [ - "Install Chroma's python client SDK, `chromadb`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "a0OB7mnI8KMw" - }, - "outputs": [], - "source": [ - "!pip install -q chromadb" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "ofnYCgA8ojgz" - }, - "source": [ - "### Grab an API Key\n", - "\n", - "To use Gemini you need an *API key*. You can create an API key with one click in [Google AI Studio](https://makersuite.google.com/).\n", - "After creating the API key, you can either set an environment variable named `GOOGLE_API_KEY` to your API Key or pass the API key as an argument when using the `Gemini` class to access Google's `gemini-1.5-flash` and `gemini-1.5-pro` models or the `GeminiEmbedding` class to access Google's Generative AI embedding model using `LlamaIndex`.\n", - "\n", - "In this tutorial, you will set the variable `gemini_api_key` to configure Gemini to use your API key." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "Wp3pYcnh60vt" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Gemini API Key:··········\n" - ] - } - ], - "source": [ - "# Run this cell and paste the API key in the prompt\n", - "import os\n", - "import getpass\n", - "\n", - "gemini_api_key = getpass.getpass('Gemini API Key:')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "gSS20CpqhaDY" - }, - "source": [ - "## Basic steps\n", - "LLMs are trained offline on a large corpus of public data. Hence they cannot answer questions based on custom or private data accurately without additional context.\n", - "\n", - "If you want to make use of LLMs to answer questions based on private data, you have to provide the relevant documents as context alongside your prompt. This approach is called Retrieval Augmented Generation (RAG).\n", - "\n", - "You will use this approach to create a question-answering assistant using the Gemini text model integrated through LlamaIndex. The assistant is expected to answer questions about Google's Gemini model. To make this possible you will add more context to the assistant using data from a website.\n", - "\n", - "In this tutorial, you'll implement the two main components in a RAG-based architecture:\n", - "\n", - "1. Retriever\n", - "\n", - " Based on the user's query, the retriever retrieves relevant snippets that add context from the document. In this tutorial, the document is the website data.\n", - " The relevant snippets are passed as context to the next stage - \"Generator\".\n", - "\n", - "2. Generator\n", - "\n", - " The relevant snippets from the website data are passed to the LLM along with the user's query to generate accurate answers.\n", - "\n", - "You'll learn more about these stages in the upcoming sections while implementing the application." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "nPKvt5_5x6rH" - }, - "source": [ - "## Import the required libraries" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "JXkg7O9PJfe3" - }, - "outputs": [], - "source": [ - "from bs4 import BeautifulSoup\n", - "from IPython.display import Markdown, display\n", - "from llama_index.core import Document\n", - "from llama_index.core import Settings\n", - "from llama_index.core import SimpleDirectoryReader\n", - "from llama_index.core import StorageContext\n", - "from llama_index.core import VectorStoreIndex\n", - "from llama_index.readers.web import SimpleWebPageReader\n", - "\n", - "from llama_index.vector_stores.chroma import ChromaVectorStore\n", - "\n", - "import chromadb\n", - "import re" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "fCU_lprVhixQ" - }, - "source": [ - "## 1. Retriever\n", - "\n", - "In this stage, you will perform the following steps:\n", - "\n", - "1. Read and parse the website data using LlamaIndex.\n", - "\n", - "2. Create embeddings of the website data.\n", - "\n", - " Embeddings are numerical representations (vectors) of text. Hence, text with similar meaning will have similar embedding vectors. You'll make use of Gemini's embedding model to create the embedding vectors of the website data.\n", - "\n", - "3. Store the embeddings in Chroma's vector store.\n", - " \n", - " Chroma is a vector database. The Chroma vector store helps in the efficient retrieval of similar vectors. Thus, for adding context to the prompt for the LLM, relevant embeddings of the text matching the user's question can be retrieved easily using Chroma.\n", - "\n", - "4. Create a Retriever from the Chroma vector store.\n", - "\n", - " The retriever will be used to pass relevant website embeddings to the LLM along with user queries." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "FergxGcKh_b_" - }, - "source": [ - "### Read and parse the website data\n", - "\n", - "LlamaIndex provides a wide variety of data loaders. To read the website data as a document, you will use the `SimpleWebPageReader` from LlamaIndex.\n", - "\n", - "To know more about how to read and parse input data from different sources using the data loaders of LlamaIndex, read LlamaIndex's [loading data guide](https://docs.llamaindex.ai/en/stable/understanding/loading/loading.html)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "xIYUiPPNjMrr" - }, - "outputs": [], - "source": [ - "web_documents = SimpleWebPageReader().load_data(\n", - " [\"https://blog.google/technology/ai/google-gemini-ai/\"]\n", - ")\n", - "\n", - "# Extract the content from the website data document\n", - "html_content = web_documents[0].text" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "TamoAP7ckyvB" - }, - "source": [ - "You can use variety of HTML parsers to extract the required text from the html content.\n", - "\n", - "In this example, you'll use Python's `BeautifulSoup` library to parse the website data. After processing, the extracted text should be converted back to LlamaIndex's `Document` format." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "-90BtEGikzt1" - }, - "outputs": [], - "source": [ - "# Parse the data.\n", - "soup = BeautifulSoup(html_content, 'html.parser')\n", - "p_tags = soup.findAll('p')\n", - "text_content = \"\"\n", - "for each in p_tags:\n", - " text_content += each.text + \"\\n\"\n", - "\n", - "# Convert back to Document format\n", - "documents = [Document(text=text_content)]" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "sq-MBiAgw1ba" - }, - "source": [ - "### Initialize Gemini's embedding model\n", - "\n", - "To create the embeddings from the website data, you'll use Gemini's embedding model, **embedding-001** which supports creating text embeddings.\n", - "\n", - "To use this embedding model, you have to import `GeminiEmbedding` from LlamaIndex. To know more about the embedding model, read Google AI's [language documentation](https://ai.google.dev/models/gemini)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "Ezv0-TIiFkxv" - }, - "outputs": [], - "source": [ - "from llama_index.embeddings.gemini import GeminiEmbedding\n", - "\n", - "gemini_embedding_model = GeminiEmbedding(api_key=gemini_api_key, model_name=\"models/embedding-001\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "vJB_fdQmoq_6" - }, - "source": [ - "### Initialize Gemini\n", - "\n", - "You must import `Gemini` from LlamaIndex to initialize your model.\n", - " In this example, you will use **gemini-pro**, as it supports text summarization. To know more about the text model, read Google AI's [model documentation](https://ai.google.dev/models/gemini).\n", - "\n", - "You can configure the model parameters such as ***temperature*** or ***top_p***, using the ***generation_config*** parameter when initializing the `Gemini` LLM. To learn more about the model parameters and their uses, read Google AI's [concepts guide](https://ai.google.dev/docs/concepts#model_parameters)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "Gyq6YIh97quL" - }, - "outputs": [], - "source": [ - "from llama_index.llms.gemini import Gemini\n", - "\n", - "# To configure model parameters use the `generation_config` parameter.\n", - "# eg. generation_config = {\"temperature\": 0.7, \"topP\": 0.8, \"topK\": 40}\n", - "# If you only want to set a custom temperature for the model use the\n", - "# \"temperature\" parameter directly.\n", - "\n", - "llm = Gemini(api_key=gemini_api_key, model_name=\"models/gemini-pro\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "uNLuJ-TY4utI" - }, - "source": [ - "### Store the data using Chroma\n", - "\n", - " Next, you'll store the embeddings of the website data in Chroma's vector store using LlamaIndex.\n", - "\n", - " First, you have to initiate a Python client in `chromadb`. Since the plan is to save the data to the disk, you will use the `PersistentClient`. You can read more about the different clients in Chroma in the [client reference guide](https://docs.trychroma.com/reference/Client).\n", - "\n", - "After initializing the client, you have to create a Chroma collection. You'll then initialize the `ChromaVectorStore` class in LlamaIndex using the collection created in the previous step.\n", - "\n", - "Next, you have to set `Settings` and create storage contexts for the vector store.\n", - "\n", - "`Settings` is a collection of commonly used resources that are utilized during the indexing and querying phase in a LlamaIndex pipeline. You can specify the LLM, Embedding model, etc that will be used to create the application in the `Settings`. To know more about `Settings`, read the [module guide for Settings](https://docs.llamaindex.ai/en/stable/module_guides/supporting_modules/settings.html).\n", - "\n", - "`StorageContext` is an abstraction offered by LlamaIndex around different types of storage. To know more about storage context, read the [storage context API guide](https://docs.llamaindex.ai/en/stable/api_reference/storage.html).\n", - "\n", - "The final step is to load the documents and build an index over them. LlamaIndex offers several indices that help in retrieving relevant context for a user query. Here you'll use the `VectorStoreIndex` since the website embeddings have to be stored in a vector store.\n", - "\n", - "To create the index you have to pass the storage context along with the documents to the `from_documents` function of `VectorStoreIndex`.\n", - "The `VectorStoreIndex` uses the embedding model specified in the `Settings` to create embedding vectors from the documents and stores these vectors in the vector store specified in the storage context. To know more about the\n", - "`VectorStoreIndex` you can read the [Using VectorStoreIndex guide](https://docs.llamaindex.ai/en/stable/module_guides/indexing/vector_store_index.html)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "1Ohzkf-LJyHO" - }, - "outputs": [], - "source": [ - "# Create a client and a new collection\n", - "client = chromadb.PersistentClient(path=\"./chroma_db\")\n", - "chroma_collection = client.get_or_create_collection(\"quickstart\")\n", - "\n", - "# Create a vector store\n", - "vector_store = ChromaVectorStore(chroma_collection=chroma_collection)\n", - "\n", - "# Create a storage context\n", - "storage_context = StorageContext.from_defaults(vector_store=vector_store)\n", - "\n", - "# Set Global settings\n", - "Settings.llm = llm\n", - "Settings.embed_model = gemini_embedding_model\n", - "\n", - "# Create an index from the documents and save it to the disk.\n", - "index = VectorStoreIndex.from_documents(\n", - " documents, storage_context=storage_context\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "ir5pUZpNu3ly" - }, - "source": [ - "### Create a retriever using Chroma\n", - "\n", - "You'll now create a retriever that can retrieve data embeddings from the newly created Chroma vector store.\n", - "\n", - "First, initialize the `PersistentClient` with the same path you specified while creating the Chroma vector store. You'll then retrieve the collection `\"quickstart\"` you created previously from Chroma. You can use this collection to initialize the `ChromaVectorStore` in which you store the embeddings of the website data. You can then use the `from_vector_store` function of `VectorStoreIndex` to load the index." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "FlAPuVLt4mBr" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "MMLU (massive multitask language understanding) is a benchmark that uses a combination of 57 subjects such as math, physics, history, law, medicine and ethics for testing both world knowledge and problem-solving abilities.\n" - ] - } - ], - "source": [ - "# Load from disk\n", - "load_client = chromadb.PersistentClient(path=\"./chroma_db\")\n", - "\n", - "# Fetch the collection\n", - "chroma_collection = load_client.get_collection(\"quickstart\")\n", - "\n", - "# Fetch the vector store\n", - "vector_store = ChromaVectorStore(chroma_collection=chroma_collection)\n", - "\n", - "# Get the index from the vector store\n", - "index = VectorStoreIndex.from_vector_store(\n", - " vector_store\n", - ")\n", - "\n", - "# Check if the retriever is working by trying to fetch the relevant docs related\n", - "# to the phrase 'MMLU' (Multimodal Machine Learning Understanding).\n", - "# If the length is greater than zero, it means that the retriever is\n", - "# functioning well.\n", - "# You can ask questions about your data using a generic interface called\n", - "# a query engine. You have to use the `as_query_engine` function of the\n", - "# index to create a query engine and use the `query` function of query engine\n", - "# to inquire the index.\n", - "test_query_engine = index.as_query_engine()\n", - "response = test_query_engine.query(\"MMLU\")\n", - "print(response)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "10heqY7ilEsi" - }, - "source": [ - "## 2. Generator\n", - "\n", - "The Generator prompts the LLM for an answer when the user asks a question. The retriever you created in the previous stage from the Chroma vector store will be used to pass relevant embeddings from the website data to the LLM to provide more context to the user's query.\n", - "\n", - "You'll perform the following steps in this stage:\n", - "\n", - "1. Create a prompt for answering any question using LlamaIndex.\n", - " \n", - "2. Use a query engine to ask a question and prompt the model for an answer." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "iCLTx4zSxSll" - }, - "source": [ - "### Create prompt templates\n", - "\n", - "You'll use LlamaIndex's [PromptTemplate](https://docs.llamaindex.ai/en/stable/module_guides/models/prompts.html) to generate prompts to the LLM for answering questions.\n", - "\n", - "In the `llm_prompt`, the variable `query_str` will be replaced later by the input question, and the variable `context_str` will be replaced by the relevant text from the website retrieved from the Chroma vector store." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "V96dQi1uOzfr" - }, - "outputs": [], - "source": [ - "from llama_index.core import PromptTemplate\n", - "\n", - "template = (\n", - " \"\"\" You are an assistant for question-answering tasks.\n", - "Use the following context to answer the question.\n", - "If you don't know the answer, just say that you don't know.\n", - "Use five sentences maximum and keep the answer concise.\\n\n", - "Question: {query_str} \\nContext: {context_str} \\nAnswer:\"\"\"\n", - ")\n", - "llm_prompt = PromptTemplate(template)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "-aE0YWHT7bal" - }, - "source": [ - "### Prompt the model using Query Engine\n", - "\n", - "You will use the `as_query_engine` function of the `VectorStoreIndex` to create a query engine from the index using the `llm_prompt` passed as the value for the `text_qa_template` argument. You can then use the `query` function of the query engine to prompt the LLM. To know more about custom prompting in LlamaIndex, read LlamaIndex's [prompts usage pattern documentation](https://docs.llamaindex.ai/en/stable/module_guides/models/prompts/usage_pattern.html#defining-a-custom-prompt)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "klNUEBbP3xbr" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Gemini is the most capable and general model that Google has ever built. It is a multimodal AI model that can understand and generate text, images, and code. Gemini is being used to power new features in a range of Google products, including Bard, Pixel, Search, and Ads.\n" - ] - } - ], - "source": [ - "# Query data from the persisted index\n", - "query_engine = index.as_query_engine(text_qa_template=llm_prompt)\n", - "response = query_engine.query(\"What is Gemini?\")\n", - "print(response)" - ] - } - ], - "metadata": { - "colab": { - "name": "Gemini_LlamaIndex_QA_Chroma_WebPageReader.ipynb", - "toc_visible": true - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} diff --git a/examples/gemini/python/vectordb_with_chroma/vectordb_with_chroma.ipynb b/examples/gemini/python/vectordb_with_chroma/vectordb_with_chroma.ipynb deleted file mode 100644 index 77f84cc5d..000000000 --- a/examples/gemini/python/vectordb_with_chroma/vectordb_with_chroma.ipynb +++ /dev/null @@ -1,818 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "Tce3stUlHN0L" - }, - "source": [ - "##### Copyright 2023 Google LLC." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "tuOe1ymfHZPu" - }, - "outputs": [], - "source": [ - "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", - "# you may not use this file except in compliance with the License.\n", - "# You may obtain a copy of the License at\n", - "#\n", - "# https://www.apache.org/licenses/LICENSE-2.0\n", - "#\n", - "# Unless required by applicable law or agreed to in writing, software\n", - "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", - "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", - "# See the License for the specific language governing permissions and\n", - "# limitations under the License." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "CsVPnR8VbXE6" - }, - "source": [ - "# Document Q&A with ChromaDB" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "awKO767lQIWh" - }, - "source": [ - "\n", - " \n", - " \n", - "
\n", - " Run in Google Colab\n", - " \n", - " View source on GitHub\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "YtwZ8DZGJfUv" - }, - "source": [ - "## Overview\n", - "\n", - "This tutorial demonstrates how to use the Gemini API to create a vector database and retrieve answers to questions from the database. Moreover, you will use [ChromaDB](https://docs.trychroma.com/){:.external}, an open-source Python tool that creates embedding databases. ChromaDB allows you to:\n", - "\n", - "* Store embeddings as well as their metadata\n", - "* Embed documents and queries\n", - "* Search through the database of embeddings\n", - "\n", - "In this tutorial, you'll use embeddings to retrieve an answer from a database of vectors created with ChromaDB.\n", - "\n", - "## Prerequisites\n", - "\n", - "You can run this quickstart in Google Colab.\n", - "\n", - "To complete this quickstart on your own development environment, ensure that your environment meets the following requirements:\n", - "\n", - "- Python 3.9+\n", - "- An installation of `jupyter` to run the notebook." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "akuOzK4dJl3j" - }, - "source": [ - "## Setup\n", - "\n", - "First, download and install ChromaDB and the Gemini API Python library." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "JbXe7Oodc5dP" - }, - "outputs": [], - "source": [ - "!pip install -U -q google-generativeai" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "sNCv-cJPLOZ2" - }, - "outputs": [], - "source": [ - "!pip install -q chromadb" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "jwmKt115PxK8" - }, - "source": [ - "Then import the modules you'll use in this tutorial." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "muuhsDmmKdHi" - }, - "outputs": [], - "source": [ - "import textwrap\n", - "import chromadb\n", - "import numpy as np\n", - "import pandas as pd\n", - "\n", - "import google.generativeai as genai\n", - "\n", - "# Used to securely store your API key\n", - "from google.colab import userdata\n", - "\n", - "from IPython.display import Markdown\n", - "from chromadb import Documents, EmbeddingFunction, Embeddings" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "U6tZGHUDOCFW" - }, - "source": [ - "### Grab an API Key\n", - "\n", - "Before you can use the Gemini API, you must first obtain an API key. If you don't already have one, create a key with one click in Google AI Studio.\n", - "\n", - "Get an API key\n", - "\n", - "In Colab, add the key to the secrets manager under the \"🔑\" in the left panel. Give it the name `API_KEY`.\n", - "\n", - "Once you have the API key, pass it to the SDK. You can do this in two ways:\n", - "\n", - "* Put the key in the `GOOGLE_API_KEY` environment variable (the SDK will automatically pick it up from there).\n", - "* Pass the key to `genai.configure(api_key=...)`" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "JoCFT6SaiCBX" - }, - "outputs": [], - "source": [ - "# Or use `os.getenv('API_KEY')` to fetch an environment variable.\n", - "API_KEY=userdata.get('API_KEY')\n", - "\n", - "genai.configure(api_key=API_KEY)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "fegnGFpMS4AI" - }, - "source": [ - "Key Point: Next, you will choose a model. Any embedding model will work for this tutorial, but for real applications it's important to choose a specific model and stick with it. The outputs of different models are not compatible with each other.\n", - "\n", - "**Note**: At this time, the Gemini API is [only available in certain regions](https://ai.google.dev/available_regions)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "Km5d13_FS2Q_" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "models/embedding-001\n", - "models/embedding-001\n" - ] - } - ], - "source": [ - "for m in genai.list_models():\n", - " if 'embedContent' in m.supported_generation_methods:\n", - " print(m.name)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "3XWKXoXwOGxS" - }, - "source": [ - "### Data\n", - "\n", - "Here is a small set of documents you will use to create an embedding database:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "k8nsbhFJKmG-" - }, - "outputs": [], - "source": [ - "DOCUMENT1 = \"Operating the Climate Control System Your Googlecar has a climate control system that allows you to adjust the temperature and airflow in the car. To operate the climate control system, use the buttons and knobs located on the center console. Temperature: The temperature knob controls the temperature inside the car. Turn the knob clockwise to increase the temperature or counterclockwise to decrease the temperature. Airflow: The airflow knob controls the amount of airflow inside the car. Turn the knob clockwise to increase the airflow or counterclockwise to decrease the airflow. Fan speed: The fan speed knob controls the speed of the fan. Turn the knob clockwise to increase the fan speed or counterclockwise to decrease the fan speed. Mode: The mode button allows you to select the desired mode. The available modes are: Auto: The car will automatically adjust the temperature and airflow to maintain a comfortable level. Cool: The car will blow cool air into the car. Heat: The car will blow warm air into the car. Defrost: The car will blow warm air onto the windshield to defrost it.\"\n", - "DOCUMENT2 = \"Your Googlecar has a large touchscreen display that provides access to a variety of features, including navigation, entertainment, and climate control. To use the touchscreen display, simply touch the desired icon. For example, you can touch the \\\"Navigation\\\" icon to get directions to your destination or touch the \\\"Music\\\" icon to play your favorite songs.\"\n", - "DOCUMENT3 = \"Shifting Gears Your Googlecar has an automatic transmission. To shift gears, simply move the shift lever to the desired position. Park: This position is used when you are parked. The wheels are locked and the car cannot move. Reverse: This position is used to back up. Neutral: This position is used when you are stopped at a light or in traffic. The car is not in gear and will not move unless you press the gas pedal. Drive: This position is used to drive forward. Low: This position is used for driving in snow or other slippery conditions.\"\n", - "\n", - "documents = [DOCUMENT1, DOCUMENT2, DOCUMENT3]" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "yDzxArLeOexD" - }, - "source": [ - "## Creating the embedding database with ChromaDB\n", - "\n", - "You will create a [custom function](https://docs.trychroma.com/embeddings#custom-embedding-functions){:.external} for performing embedding using the Gemini API. By inputting a set of documents into this custom function, you will receive vectors, or embeddings of the documents.\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "UoHhS32txd_r" - }, - "source": [ - "### API changes to Embeddings with model embedding-001\n", - "\n", - "For the new embeddings model, embedding-001, there is a new task type parameter and the optional title (only valid with task_type=`RETRIEVAL_DOCUMENT`).\n", - "\n", - "These new parameters apply only to the newest embeddings models.The task types are:\n", - "\n", - "Task Type | Description\n", - "--- | ---\n", - "RETRIEVAL_QUERY\t| Specifies the given text is a query in a search/retrieval setting.\n", - "RETRIEVAL_DOCUMENT | Specifies the given text is a document in a search/retrieval setting.\n", - "SEMANTIC_SIMILARITY\t| Specifies the given text will be used for Semantic Textual Similarity (STS).\n", - "CLASSIFICATION\t| Specifies that the embeddings will be used for classification.\n", - "CLUSTERING\t| Specifies that the embeddings will be used for clustering." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "mF7Uu1kCQsT0" - }, - "outputs": [], - "source": [ - "class GeminiEmbeddingFunction(EmbeddingFunction):\n", - " def __call__(self, input: Documents) -> Embeddings:\n", - " model = 'models/embedding-001'\n", - " title = \"Custom query\"\n", - " return genai.embed_content(model=model,\n", - " content=input,\n", - " task_type=\"retrieval_document\",\n", - " title=title)[\"embedding\"]" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "HrDWLyopPNBf" - }, - "source": [ - "Now you will create the vector database. In the `create_chroma_db` function, you will instantiate a [Chroma client](https://docs.trychroma.com/getting-started){:.external}. From there, you will create a collection, which is where you store your embeddings, documents, and any metadata. Note that the embedding function from above is passed as an argument to the `create_collection`.\n", - "\n", - "Next, you use the `add` method to add the documents to the collection." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "OITXgxZlLoXU" - }, - "outputs": [], - "source": [ - "def create_chroma_db(documents, name):\n", - " chroma_client = chromadb.Client()\n", - " db = chroma_client.create_collection(name=name, embedding_function=GeminiEmbeddingFunction())\n", - "\n", - " for i, d in enumerate(documents):\n", - " db.add(\n", - " documents=d,\n", - " ids=str(i)\n", - " )\n", - " return db" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "RJ3Fq0yzL10B" - }, - "outputs": [], - "source": [ - "# Set up the DB\n", - "db = create_chroma_db(documents, \"googlecarsdatabase\")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "2QbwFgfXp-fL" - }, - "source": [ - "Confirm that the data was inserted by looking at the database:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "kQ9PHUL_l-hf" - }, - "outputs": [ - { - "data": { - "application/vnd.google.colaboratory.intrinsic+json": { - "type": "dataframe" - }, - "text/html": [ - "\n", - "
\n", - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
idsembeddingsmetadatasdocumentsurisdata
00[-0.020994942635297775, -0.03876612335443497, ...NoneOperating the Climate Control System Your Goo...NoneNone
11[0.017410801723599434, -0.04757162556052208, -...NoneYour Googlecar has a large touchscreen display...NoneNone
22[-0.03194405511021614, -0.023281503468751907, ...NoneShifting Gears Your Googlecar has an automatic...NoneNone
\n", - "
\n", - "
\n", - "\n", - "
\n", - " \n", - "\n", - " \n", - "\n", - " \n", - "
\n", - "\n", - "\n", - "
\n", - " \n", - "\n", - "\n", - "\n", - " \n", - "
\n", - "\n", - "
\n", - "
\n" - ], - "text/plain": [ - " ids embeddings metadatas \\\n", - "0 0 [-0.020994942635297775, -0.03876612335443497, ... None \n", - "1 1 [0.017410801723599434, -0.04757162556052208, -... None \n", - "2 2 [-0.03194405511021614, -0.023281503468751907, ... None \n", - "\n", - " documents uris data \n", - "0 Operating the Climate Control System Your Goo... None None \n", - "1 Your Googlecar has a large touchscreen display... None None \n", - "2 Shifting Gears Your Googlecar has an automatic... None None " - ] - }, - "execution_count": 108, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "pd.DataFrame(db.peek(3))" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Tu5zRErgsQ8u" - }, - "source": [ - "## Getting the relevant document\n", - "\n", - "`db` is a Chroma collection object. You can call `query` on it to perform a nearest neighbors search to find similar embeddings or documents.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "gQdJMbTSLtKE" - }, - "outputs": [], - "source": [ - "def get_relevant_passage(query, db):\n", - " passage = db.query(query_texts=[query], n_results=1)['documents'][0][0]\n", - " return passage" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "nWYXXKJ6t6Hy" - }, - "outputs": [ - { - "data": { - "text/markdown": [ - "Your Googlecar has a large touchscreen display that provides access to a variety of features, including navigation, entertainment, and climate control. To use the touchscreen display, simply touch the desired icon. For example, you can touch the \"Navigation\" icon to get directions to your destination or touch the \"Music\" icon to play your favorite songs." - ], - "text/plain": [ - "" - ] - }, - "execution_count": 112, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Perform embedding search\n", - "passage = get_relevant_passage(\"touch screen features\", db)\n", - "Markdown(passage)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "s8PNRMpOQkm5" - }, - "source": [ - "Now that you have found the relevant passage in your set of documents, you can use it make a prompt to pass into the Gemini API." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "Qkhu4iazLy3G" - }, - "outputs": [], - "source": [ - "def make_prompt(query, relevant_passage):\n", - " escaped = relevant_passage.replace(\"'\", \"\").replace('\"', \"\").replace(\"\\n\", \" \")\n", - " prompt = (\"\"\"You are a helpful and informative bot that answers questions using text from the reference passage included below. \\\n", - " Be sure to respond in a complete sentence, being comprehensive, including all relevant background information. \\\n", - " However, you are talking to a non-technical audience, so be sure to break down complicated concepts and \\\n", - " strike a friendly and converstional tone. \\\n", - " If the passage is irrelevant to the answer, you may ignore it.\n", - " QUESTION: '{query}'\n", - " PASSAGE: '{relevant_passage}'\n", - "\n", - " ANSWER:\n", - " \"\"\").format(query=query, relevant_passage=escaped)\n", - "\n", - " return prompt" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "hMEjbz4EswQ6" - }, - "source": [ - "Pass a query to the prompt:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "b6_Y-GOymaXu" - }, - "outputs": [ - { - "data": { - "text/markdown": [ - "You are a helpful and informative bot that answers questions using text from the reference passage included below. Be sure to respond in a complete sentence, being comprehensive, including all relevant background information. However, you are talking to a non-technical audience, so be sure to break down complicated concepts and strike a friendly and converstional tone. If the passage is irrelevant to the answer, you may ignore it.\n", - " QUESTION: 'How do you shift gears in the Google car?'\n", - " PASSAGE: 'Your Googlecar has a large touchscreen display that provides access to a variety of features, including navigation, entertainment, and climate control. To use the touchscreen display, simply touch the desired icon. For example, you can touch the Navigation icon to get directions to your destination or touch the Music icon to play your favorite songs.'\n", - "\n", - " ANSWER:\n", - " " - ], - "text/plain": [ - "" - ] - }, - "execution_count": 117, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "query = \"How do you use the touchscreen in the Google car?\"\n", - "prompt = make_prompt(query, passage)\n", - "Markdown(prompt)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "VRy6yXzcPxLB" - }, - "source": [ - "Now use the `generate_content` method to to generate a response from the model." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "EwfyxFM6Giy9" - }, - "outputs": [], - "source": [ - "model = genai.GenerativeModel('gemini-pro')\n", - "answer = model.generate_content(prompt)\n", - "Markdown(answer.text)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "ThTbjAJ7eGP5" - }, - "source": [ - "## Next steps\n", - "\n", - "To learn more about how you can use the embeddings, check out the [examples](https://ai.google.dev/examples?keywords=embed) available. To learn how to use other services in the Gemini API, visit the [Python quickstart](https://ai.google.dev/gemini-api/docs/get-started/python)." - ] - } - ], - "metadata": { - "colab": { - "name": "vectordb_with_chroma.ipynb", - "toc_visible": true - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} diff --git a/examples/gemini/python/vectordb_with_qdrant/Qdrant_similarity_search.ipynb b/examples/gemini/python/vectordb_with_qdrant/Qdrant_similarity_search.ipynb deleted file mode 100644 index 5d9ca0ee2..000000000 --- a/examples/gemini/python/vectordb_with_qdrant/Qdrant_similarity_search.ipynb +++ /dev/null @@ -1,535 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "lTx8eQlc3cP-" - }, - "source": [ - "##### Copyright 2024 Google LLC." - ] - }, - { - "cell_type": "code", - "execution_count": 1, - "metadata": { - "cellView": "form", - "id": "4HZoi8yf4GEU" - }, - "outputs": [], - "source": [ - "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", - "# you may not use this file except in compliance with the License.\n", - "# You may obtain a copy of the License at\n", - "#\n", - "# https://www.apache.org/licenses/LICENSE-2.0\n", - "#\n", - "# Unless required by applicable law or agreed to in writing, software\n", - "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", - "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", - "# See the License for the specific language governing permissions and\n", - "# limitations under the License." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "M9I7LG483nXB" - }, - "source": [ - "# Similarity Search using Gemini API and Qdrant" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "awKO767lQIWh" - }, - "source": [ - "\n", - " \n", - " \n", - "
\n", - " Run in Google Colab\n", - " \n", - " View source on GitHub\n", - "
\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "b1xoF_bU4NCP" - }, - "source": [ - "## Overview" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "CWedABji6bXJ" - }, - "source": [ - "The [Gemini API](https://ai.google.dev/models/gemini) provides access to a family of generative AI models for generating content and solving problems. These models are designed and trained to handle both text and images as input.\n", - "\n", - "[Qdrant](https://qdrant.tech/) is a vector similarity search engine that offers an easy-to-use API for managing, storing, and searching vectors, with an additional payload. It is a production-ready service.\n", - "\n", - "In this notebook, you'll learn how to perform a similarity search on data from a website with the help of Gemini API and Qdrant." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "dIAarGkG8VwC" - }, - "source": [ - "## Setup\n", - "\n", - "First, you must install the packages and set the necessary environment variables.\n", - "\n", - "### Installation\n", - "\n", - "Install google's python client SDK for the Gemini API, `google-generativeai`." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "id": "LnvqwC7AFROK" - }, - "outputs": [], - "source": [ - "! pip install -q google-generativeai" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "70wYOKUC8q1m" - }, - "source": [ - "Install Qdrant's python client SDK, `qdrant-client`." - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "id": "mnQbBnA1GKha" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m229.3/229.3 kB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.8/2.8 MB\u001b[0m \u001b[31m17.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.6/75.6 kB\u001b[0m \u001b[31m4.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m294.6/294.6 kB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m77.9/77.9 kB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.5/57.5 kB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", - "\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n", - "tensorflow-metadata 1.14.0 requires protobuf<4.21,>=3.20.3, but you have protobuf 4.25.3 which is incompatible.\u001b[0m\u001b[31m\n", - "\u001b[0m" - ] - } - ], - "source": [ - "! pip install -q qdrant-client" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "RzppByiY85Uc" - }, - "source": [ - "### Grab and set the API key\n", - "\n", - "To use Gemini API you need an *API key*. You can create an API key with one click in [Google AI Studio](https://makersuite.google.com/).\n", - "\n", - "Once you have the API key, pass it to the SDK. You can do this in two ways:\n", - "\n", - "1. Assign the key to the `GOOGLE_API_KEY` environment variable (the SDK will automatically pick it up from there) or pass the key to `genai.configure(api_key=...)`.\n", - "2. Or provide it explicitly through the `api_key` parameter.\n", - "\n", - "To run the following cell, your API key must be stored it in a Colab Secret named `GOOGLE_API_KEY`. If you don't already have an API key, or you're not sure how to create a Colab Secret, see the [Authentication](https://github.com/google-gemini/cookbook/blob/main/quickstarts/Authentication.ipynb) guide for an example." - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "id": "MWn09K5G8XYZ" - }, - "outputs": [], - "source": [ - "import google.generativeai as genai" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "id": "MWn09K5G87YX" - }, - "outputs": [], - "source": [ - "from google.colab import userdata\n", - "GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')\n", - "\n", - "genai.configure(api_key=GOOGLE_API_KEY)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "-go1BAF-_GtV" - }, - "source": [ - "## Basic steps\n", - "\n", - "Semantic search is the process using which search engines interpret and match keywords to a user's intent in organic search results. It goes beyond surface-level keyword matching. It uses the meaning of words, phrases, and context using advanced algorithms resulting in more relevant and user-friendly search experiences.\n", - "\n", - "Semantic searches rely on vector embeddings which can best match the user query to the most similar result.\n", - "\n", - "In this tutorial, you'll implement the three main components of semantic search:\n", - "\n", - "1. Create an index\n", - "\n", - " Create and store the index for the data in the Qdrant vector store. You will use a Gemini API embedding model to create embedding vectors that can be stored in the Qdrant vector store.\n", - "\n", - "2. Query the index\n", - "\n", - " Query the index using a query string to return the top `n` neighbors of the query.\n", - "\n", - "You'll learn more about these stages in the upcoming sections while implementing the application." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "0egnCR92JKsj" - }, - "source": [ - "## Import the required libraries" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "id": "LfJN5QosJQqD" - }, - "outputs": [], - "source": [ - "from bs4 import BeautifulSoup\n", - "from qdrant_client import models, QdrantClient\n", - "from urllib.request import urlopen" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "lL7J7BtyJsNQ" - }, - "source": [ - "## 1. Create an index\n", - "\n", - "In this stage, you will perform the following steps:\n", - "\n", - "1. Read and parse the website data using Python's BeautifulSoup library.\n", - "\n", - "2. Create embeddings of the website data.\n", - "\n", - "3. Store the embeddings in Qdrant's vector database.\n", - " \n", - " Qdrant is a vector similarity search engine. Along with a convenient API to store, search, and manage points(i.e. vectors), it also provides an option to add an additional payload. The payloads are essentially extra bits of data that you can utilize to refine your search and obtain relevant information that you can then share with your users." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "kFlGmkKbRebP" - }, - "source": [ - "### Read and parse the website data\n", - "\n", - "To read the website data as text, you will use the `BeautifulSoup` library from Python." - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "id": "oMs-ux1gtxOa" - }, - "outputs": [], - "source": [ - "url = \"https://blog.google/outreach-initiatives/sustainability/\"\\\n", - " \"report-ai-sustainability-google-cop28/\"\n", - "html = urlopen(url).read()\n", - "soup = BeautifulSoup(html, features=\"html.parser\")\n", - "\n", - "# Remove all script and style elements\n", - "for script in soup([\"script\", \"style\"]):\n", - " script.extract() # Self-destruct\n", - "\n", - "# Get the text\n", - "text_content = soup.get_text()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "toC03rqUSfom" - }, - "source": [ - "If you only want to select a specific portion of the website data to add context to the prompt, you can use regex, text slicing, or text splitting.\n", - "\n", - "In this example, you'll use Python's `split()` function to extract the required portion of the text." - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "id": "cHJq059duxj7" - }, - "outputs": [], - "source": [ - "# The text content between the substrings \"Later this month at COP28\" to\n", - "# \"POSTED IN:\" is relevant for this tutorial. You can use Python's `split()`\n", - "# to select the required content.\n", - "text_content_1 = text_content.split(\"Later this month at COP28\",1)[1]\n", - "final_text = text_content_1.split(\"POSTED IN:\",1)[0]\n", - "\n", - "texts = final_text.split(\".\")\n", - "\n", - "documents = []\n", - "\n", - "# Convert text into a chunk of 3 sentences.\n", - "for i in range(0, len(texts), 3):\n", - " documents.append({\"content\": \" \".join(texts[i:i+3])})" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "-CVPdm0h6aTd" - }, - "source": [ - "### Initialize the embedding model\n", - "\n", - "To create the embeddings from the website data, you'll use the **embedding-001** model, which supports creating embeddings from text.\n", - "\n", - "To use the embedding model, you have to use the `embed_content` function from the `google-generativeai` package. To learn more about the embedding model, read the [model documentation](https://ai.google.dev/gemini-api/docs/models/gemini#embedding).\n", - "\n", - "One of the arguments passed to the embedding function is `task_type`. Speciefying the `task_type` parameter ensures the model produces appropriate embeddingsfor the expected task and inputs. It is a string that can take on one of the following values:\n", - "\n", - "| task_type\t | Description |\n", - "|---|---|\n", - "| `RETRIEVAL_QUERY` | Specifies the given text is a query in a search or retrieval setting. |\n", - "| `RETRIEVAL_DOCUMENT` | Specifies the given text is a document in a search or retrieval setting. | \n", - "| `SEMANTIC_SIMILARITY` | Specifies the given text will be used for Semantic Textual Similarity (STS). | \n", - "| `CLASSIFICATION` | Specifies that the embeddings will be used for classification. |\n", - "| `CLUSTERING` | Specifies that the embeddings will be used for clustering. |" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "id": "EeSW5NOBTS0a" - }, - "outputs": [], - "source": [ - "# Default embedding model\n", - "embedding_model = \"models/embedding-001\"\n", - "\n", - "# Function to convert text to embeddings\n", - "def make_embed_text_fn(text, model=embedding_model,\n", - " task_type=\"retrieval_document\"):\n", - " embedding = genai.embed_content(model=model,\n", - " content=text,\n", - " task_type=task_type)\n", - " return embedding['embedding']" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "9ajByZTxVXal" - }, - "source": [ - "### Store the data using Qdrant\n", - "\n", - " Next, you'll store the embeddings of the website data in Qdrant's vector store.\n", - "\n", - " First, you have to initiate a Qdrant client by creating an instance of `QdrantClient`. In this tutorial, you will store the embeddings in memory. To create an in-memory Qdrant client specify `:memory:` for the `location` argument of the `QdrantClient` class initializer. You can read more about the different types of storage in Qdrant in the [storage reference guide](https://qdrant.tech/documentation/concepts/storage/).\n", - "\n", - "After initializing the client, you have to create a Qdrant collection using the `recreate_collection` function of `QdrantClient`. You can specify your vector configuration inside the `recreate_collection` function. Pass an instance of `VectorParams` with the `size` set to `768` to match the embedding model and `distance` set to cosine.\n", - "\n", - "**Note**: Since you will run the script several times during your experiments, `recreate_collection` is appropriate for this tutorial. `recreate_collection` will first try to remove an existing collection with the same name." - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "id": "pnURtmtZTImC" - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - ":5: DeprecationWarning: `recreate_collection` method is deprecated and will be removed in the future. Use `collection_exists` to check collection existence and `create_collection` instead.\n", - " qdrant.recreate_collection(\n" - ] - }, - { - "data": { - "text/plain": [ - "True" - ] - }, - "execution_count": 9, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Initialize Qdrant client.\n", - "qdrant = QdrantClient(\":memory:\")\n", - "\n", - "# Create a collection named \"GeminiCollection\".\n", - "qdrant.recreate_collection(\n", - " collection_name=\"GeminiCollection\",\n", - " vectors_config=models.VectorParams(\n", - " size=768, # Vector size of `embedding-001`\n", - " distance=models.Distance.COSINE,\n", - " ),\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "YQu7FIhLeC0O" - }, - "source": [ - "You will now insert the `documents` you parsed from the website data into the Qdrant collection you created earlier and index them using the `upsert` function of `QdrantClient`.\n", - "\n", - "The `upsert` function takes the data to be stored and indexed as an array of `PointsStruct`s.\n", - "\n", - "Points are the main entity in Qdrant operations. A point is a record consisting of a vector and an optional payload. You can perform a similarity search among the points in one collection. Read more about points in [Qdrant's points documentation](https://qdrant.tech/documentation/concepts/points/).\n", - "\n", - "You'll create an array of points by enumerating over the documents you prepared earlier from the website data." - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": { - "id": "uOqivudxSyR9" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "UpdateResult(operation_id=0, status=)" - ] - }, - "execution_count": 10, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Qdrant uses batch loading of points to optimize performance.\n", - "# You can create a batch in two ways - record-oriented and column-oriented.\n", - "# Here you are using the record-oriented approach.\n", - "\n", - "qdrant.upsert(\n", - " collection_name=\"GeminiCollection\",\n", - " points=[\n", - " # Use PointStruct function to intialize the point.\n", - " models.PointStruct(\n", - " # Use `make_embed_text_fn` to convert text to embeddings.\n", - " # Pass the same data as payload for a refined search.\n", - " id=idx, vector=make_embed_text_fn(doc[\"content\"]), payload = doc\n", - " )\n", - " for idx, doc in enumerate(documents)\n", - " ]\n", - ")" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "JdVrKZZ0cTkV" - }, - "source": [ - "## 2. Query the index\n", - "\n", - "You'll now query the Qdrant index you created earlier with a question related to the data contained in the website documents.\n", - "To query the index, you have to mention the collection name and the query vector. The query vector should be first converted to an embedding vector using the Gemini API embedding model you leveraged to create embedding vectors for the website data. Use the `make_embed_text_fn` you defined earlier for creating an embedding vector from your query. Since you are embedding a query string that is being used to search `retrieval_document` embeddings, the `task_type` must be set to `retrieval_query`." - ] - }, - { - "cell_type": "code", - "execution_count": 11, - "metadata": { - "id": "6LQVKNfMTyOx" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "{'content': ' Already, it is starting to address climate challenges in three key areas: providing people and organizations with better information to make more sustainable choices, delivering improved predictions to help adapt to climate change, and finding recommendations to optimize climate action for high-impact applications Here’s a look at how, at Google, we’ve used AI to address climate challenges:Providing helpful information: People are looking for information to reduce their environmental footprint Fuel-efficient routing in Google Maps uses AI to suggest routes that have fewer hills, less traffic, and constant speeds with the same or similar ETA'} score: 0.7711945535904017\n", - "{'content': ' Policymakers, in particular, have a central role to play both in harnessing the potential of AI for climate action and in ensuring its sustainable and equitable use Policymakers can make a difference in accelerating three outcomes:Enabling AI for climate progress by encouraging data sharing, ensuring affordable technology access, building awareness, and supporting the creation and expansion of AI and climate-related upskilling programs for corporations Accelerating the deployment of AI for climate by defining public and private sector priorities, delivering on public sector use cases, and encouraging private sector action'} score: 0.7458781382056137\n", - "{'content': '\\n\\n\\n\\n\\nManaging the environmental impact of AIWhile scaling these applications of AI and finding new ways to use it to accelerate climate action is crucial, we need to build AI responsibly and manage the environmental impact associated with it As AI is at an inflection point, predicting the future growth of energy use and emissions from AI compute in our data centers is challenging Historically, data center energy consumption has grown much more slowly than demand for computing power'} score: 0.7405380973240167\n" - ] - } - ], - "source": [ - "hits = qdrant.search(\n", - " collection_name=\"GeminiCollection\",\n", - " query_vector=make_embed_text_fn(\"How can AI address climate challenges?\",\n", - " task_type=\"retrieval_query\"),\n", - " limit=3,\n", - ")\n", - "for hit in hits:\n", - " print(hit.payload, \"score:\", hit.score)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Tt1wSSMIxsf2" - }, - "source": [ - "##Conclusion\n", - "\n", - "That's it. You have successfully performed a similarity search using Qdrant with the help of a Gemini API embedding model." - ] - } - ], - "metadata": { - "colab": { - "name": "Qdrant_similarity_search.ipynb", - "toc_visible": true - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -}