"
]
@@ -81,10 +81,8 @@
"id": "Tdlq6K0znh3O"
},
"source": [
- "## Overview\n",
- "\n",
"Gemma is a family of lightweight, state-of-the-art open models built from research and technology used to create Google Gemini models. Gemma can be further finetuned to suit specific needs. But Large Language Models, such as Gemma, can be very large in size and some of them may not fit on a sing accelerator for finetuning. In this case there are two general approaches for finetuning them:\n",
- "1. Parameter Efficient Fine-Tuning (PEFT), which seeks to shrink the effective model size by sacrificing some fidelity. LoRA falls in this category and the [Fine-tune Gemma models in Keras using LoRA](https://ai.google.dev/gemma/docs/lora_tuning) tutorial demonstrates how to finetune the Gemma 2B model `gemma_2b_en` with LoRA using KerasNLP on a single GPU.\n",
+ "1. Parameter Efficient Fine-Tuning (PEFT), which seeks to shrink the effective model size by sacrificing some fidelity. LoRA falls in this category and the [Fine-tune Gemma models in Keras using LoRA](https://ai.google.dev/gemma/docs/core/lora_tuning) tutorial demonstrates how to finetune the Gemma 2B model `gemma_2b_en` with LoRA using KerasNLP on a single GPU.\n",
"2. Full parameter finetuning with model parallelism. Model parallelism distributes a single model's weights across multiple devices and enables horizontal scaling. You can find out more about distributed training in this [Keras guide](https://keras.io/guides/distribution/).\n",
"\n",
"This tutorial walks you through using Keras with a JAX backend to finetune the Gemma 7B model with LoRA and model-parallism distributed training on Google's Tensor Processing Unit (TPU). Note that LoRA can be turned off in this tutorial for a slower but more accurate full-parameter tuning."
@@ -105,7 +103,7 @@
"Google has 3 products that provide TPUs:\n",
"* [Colab](https://colab.sandbox.google.com/) provides TPU v2 for free, which is sufficient for this tutorial.\n",
"* [Kaggle](https://www.kaggle.com/) offers TPU v3 for free and they also work for this tutorial.\n",
- "* [Cloud TPU](https://cloud.google.com/tpu?hl=en) offers TPU v3 and newer generations. One way to set it up is:\n",
+ "* [Cloud TPU](https://cloud.google.com/tpu) offers TPU v3 and newer generations. One way to set it up is:\n",
" 1. Create a new [TPU VM](https://cloud.google.com/tpu/docs/managing-tpus-tpu-vm#tpu-vms)\n",
" 2. Set up [SSH port forwarding](https://cloud.google.com/solutions/connecting-securely#port-forwarding-over-ssh) for your intended Jupyter server port\n",
" 3. Install Jupyter and start it on the TPU VM, then connect to Colab through \"Connect to a local runtime\"\n",
@@ -963,7 +961,7 @@
"In this tutorial, you learned how to using KerasNLP JAX backend to finetune a Gemma model on the IMDb dataset in a distributed manner on the powerful TPUs. Here are a few suggestions for what else to learn:\n",
"\n",
"* Learn how to [get started with Keras Gemma](https://ai.google.dev/gemma/docs/get_started).\n",
- "* Learn how to [finetune the Gemma model on GPU](https://ai.google.dev/gemma/docs/lora_tuning)."
+ "* Learn how to [finetune the Gemma model on GPU](https://ai.google.dev/gemma/docs/core/lora_tuning)."
]
}
],
diff --git a/site/en/gemma/docs/core/gemma_library.ipynb b/site/en/gemma/docs/core/gemma_library.ipynb
new file mode 100644
index 000000000..7f732ca5d
--- /dev/null
+++ b/site/en/gemma/docs/core/gemma_library.ipynb
@@ -0,0 +1,422 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "G3MMAcssHTML"
+ },
+ "source": [
+ "\n",
+ ""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Tce3stUlHN0L"
+ },
+ "source": [
+ "##### Copyright 2025 Google LLC."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "cellView": "form",
+ "id": "tuOe1ymfHZPu"
+ },
+ "outputs": [],
+ "source": [
+ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
+ "# you may not use this file except in compliance with the License.\n",
+ "# You may obtain a copy of the License at\n",
+ "#\n",
+ "# https://www.apache.org/licenses/LICENSE-2.0\n",
+ "#\n",
+ "# Unless required by applicable law or agreed to in writing, software\n",
+ "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
+ "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
+ "# See the License for the specific language governing permissions and\n",
+ "# limitations under the License."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "SDEExiAk4fLb"
+ },
+ "source": [
+ "# Prompt with images and text using Gemma library"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "4qxv4Sn9b8CE"
+ },
+ "source": [
+ "
"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "PXNm5_p_oxMF"
+ },
+ "source": [
+ "Using images for prompting Gemma models opens up a whole new range of possibilies for understanding your world and solving problems with visual data. Starting with [Gemma 3](/gemma/docs/core) in 4B sizes and higher, you can use image data as part of your prompt to for richer context and to solve more complex tasks.\n",
+ "\n",
+ "This tutorial shows you how to prompt Gemma with images using the [Gemma library](https://gemma-llm.readthedocs.io/) for JAX. Gemma library is a Python package built as an extension of [JAX](https://github.com/jax-ml/jax), letting you use the performance advantages of the JAX framework with dramatically less code.\n",
+ "\n",
+ "Note: For more the up-to-data information this library, see the [Gemma library](https://gemma-llm.readthedocs.io/) documentation."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "QQ6W7NzRe1VM"
+ },
+ "source": [
+ "## Setup\n",
+ "\n",
+ "To complete this tutorial, you'll first need to complete the setup instructions at [Gemma setup](https://ai.google.dev/gemma/docs/setup). The Gemma setup instructions show you how to do the following:\n",
+ "\n",
+ "* Get access to Gemma on [kaggle.com](https://www.kaggle.com).\n",
+ "* Select a Colab runtime with sufficient resources to run\n",
+ " the Gemma model.\n",
+ "* Generate and configure a Kaggle username and API key.\n",
+ "\n",
+ "After you've completed the Gemma setup, move on to the next section, where you'll set environment variables for your Colab environment."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "z9oy3QUmXtSd"
+ },
+ "source": [
+ "### Install libraries\n",
+ "\n",
+ "Install the Gemma library."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "UcGLzDeQ8NwN"
+ },
+ "outputs": [],
+ "source": [
+ "!pip install -q gemma"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "_gN-IVRC3dQe"
+ },
+ "source": [
+ "### Set environment variables\n",
+ "\n",
+ "Set environment variables for `KAGGLE_USERNAME` and `KAGGLE_KEY`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "DrBoa_Urw9Vx"
+ },
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "from google.colab import userdata\n",
+ "\n",
+ "# Note: `userdata.get` is a Colab API. If you're not using Colab, set the env\n",
+ "# vars as appropriate for your system.\n",
+ "os.environ[\"KAGGLE_USERNAME\"] = userdata.get('KAGGLE_USERNAME')\n",
+ "os.environ[\"KAGGLE_KEY\"] = userdata.get('KAGGLE_KEY')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "nbFs22cT_k_Y"
+ },
+ "source": [
+ "Set the JAX environment to use the full GPU memory space."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "Pv5gGAAg_xtQ"
+ },
+ "outputs": [],
+ "source": [
+ "os.environ[\"XLA_PYTHON_CLIENT_MEM_FRACTION\"]=\"1.00\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "599765c72722"
+ },
+ "source": [
+ "### Import packages\n",
+ "\n",
+ "Import the Gemma library and additional support libraries."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "f2fa267d75bc"
+ },
+ "outputs": [],
+ "source": [
+ "# Common imports\n",
+ "import os\n",
+ "import jax\n",
+ "import jax.numpy as jnp\n",
+ "import tensorflow_datasets as tfds\n",
+ "\n",
+ "# Gemma imports\n",
+ "from gemma import gm"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "ZsxDCbLN555T"
+ },
+ "source": [
+ "## Configure a model\n",
+ "\n",
+ "Select and configure a Gemma model for use, including a tokenizer, model architecture, and checkpoints. The Gemma libary supports all of Google's official releases of the model. You must use the `Gemma3Tokenizer` and a Gemma 3 or later model to be able to process images as part of your prompt.\n",
+ "\n",
+ "To configure the model, run the following code:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "yygIK9DEIldp"
+ },
+ "outputs": [],
+ "source": [
+ "tokenizer = gm.text.Gemma3Tokenizer()\n",
+ "\n",
+ "model = gm.nn.Gemma3_4B()\n",
+ "\n",
+ "params = gm.ckpts.load_params(gm.ckpts.CheckpointPath.GEMMA3_4B_IT)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "FOBW7piN5-sl"
+ },
+ "source": [
+ "## Generate text with text\n",
+ "\n",
+ "Start by prompting with text. The Gemma library provides a Sampler function for simple prompting."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "aae5GHrdpj2_"
+ },
+ "outputs": [],
+ "source": [
+ "sampler = gm.text.Sampler(\n",
+ " model=model,\n",
+ " params=params,\n",
+ " tokenizer=tokenizer,\n",
+ ")\n",
+ "\n",
+ "sampler.sample('Roses are red.', max_new_tokens=30)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "qH0eFH_DvYwM"
+ },
+ "source": [
+ "Change the prompt and change the maximum number of tokens to generate different output."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "vVlCnY7Gvm7U"
+ },
+ "source": [
+ "## Generate text with images\n",
+ "\n",
+ "Once you have a text prompt working, you can add images to your prompt. Make sure you have configure a Gemma 3 or later model that is 4B or higher, and configured the `Gemma3Tokenizer`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "U_gKV_cEGAQ_"
+ },
+ "source": [
+ "### Load an image\n",
+ "\n",
+ "Load an image from a data source or a local file. The following code shows how to load an image from a TensorFlow datasource:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "VEyTnnNGvgGG"
+ },
+ "outputs": [],
+ "source": [
+ "ds = tfds.data_source('oxford_flowers102', split='train')\n",
+ "image = ds[0]['image']\n",
+ "\n",
+ "# display the image\n",
+ "image"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "Jm6KW0pEI6gX"
+ },
+ "source": [
+ "### Prepare prompt with image data\n",
+ "\n",
+ "When you prompt with image data, you include a specific tag ``, to include the image with the text your prompt. You then encode the prompt with the image data using the `tokenizer` object to prepare to run it with the model."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "5eIwxSEWKI7u"
+ },
+ "outputs": [],
+ "source": [
+ "prompt = \"\"\"user\n",
+ "Describe the contents of this image.\n",
+ "\n",
+ "\n",
+ "\n",
+ "\n",
+ "model\n",
+ "\"\"\""
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "VMj0wphLKzQ1"
+ },
+ "source": [
+ "If you want to prompt with more than one image, you must include a `` tag for each image included in your prompt."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "cgwSiapfL09z"
+ },
+ "source": [
+ "### Run the prompt with image data\n",
+ "\n",
+ "After you prepare your image data and the prompt with image tags, you can run the prompt and generate output. The following code shows how to use the `Sampler` function run the prompt:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {
+ "id": "46tsdilpMIxb"
+ },
+ "outputs": [],
+ "source": [
+ "sampler = gm.text.Sampler(\n",
+ " model=model,\n",
+ " params=params,\n",
+ " tokenizer=tokenizer,\n",
+ ")\n",
+ "\n",
+ "out = sampler.sample(prompt, images=image, max_new_tokens=500)\n",
+ "print(out)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "-HUzMDoVtZOw"
+ },
+ "source": [
+ "Alternatively, you can use the `gm.text.ChatSampler()` function generate a response without requiring `` tags. For more details, see the [Gemma library for JAX](https://gemma-llm.readthedocs.io/) documentation."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {
+ "id": "z4kXWLwwMqOo"
+ },
+ "source": [
+ "## Next steps\n",
+ "\n",
+ "The Gemma library provides much more additional functionality. See these additional resources for more information:\n",
+ "\n",
+ "* [Gemma library sampling](https://gemma-llm.readthedocs.io/en/latest/colab_sampling.html)\n",
+ "* [Gemma library finetuning](https://gemma-llm.readthedocs.io/en/latest/colab_finetuning.html)\n",
+ "\n",
+ "The Gemma library for JAX provides additional functionality, including LoRA, Sharding, Quantization and more. For more details, see the [Gemma library](https://gemma-llm.readthedocs.io) documentation. If you have any feedback, or have issues using Gemma library, submit them through the repository [Issues](https://github.com/google-deepmind/gemma/issues) interface.\n"
+ ]
+ }
+ ],
+ "metadata": {
+ "accelerator": "GPU",
+ "colab": {
+ "name": "gemma_library.ipynb",
+ "toc_visible": true
+ },
+ "google": {
+ "image_path": "/site-assets/images/marketing/gemma.png",
+ "keywords": [
+ "examples",
+ "gemma",
+ "python",
+ "quickstart",
+ "text"
+ ]
+ },
+ "kernelspec": {
+ "display_name": "Python 3",
+ "name": "python3"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/site/en/gemma/docs/keras_inference.ipynb b/site/en/gemma/docs/core/keras_inference.ipynb
similarity index 97%
rename from site/en/gemma/docs/keras_inference.ipynb
rename to site/en/gemma/docs/core/keras_inference.ipynb
index bacf75f4c..cf532f3fe 100644
--- a/site/en/gemma/docs/keras_inference.ipynb
+++ b/site/en/gemma/docs/core/keras_inference.ipynb
@@ -16,7 +16,7 @@
"id": "Tce3stUlHN0L"
},
"source": [
- "##### Copyright 2024 Google LLC."
+ "##### Copyright 2025 Google LLC."
]
},
{
@@ -49,16 +49,16 @@
"source": [
"
"
]
@@ -576,8 +576,8 @@
"\n",
"In this tutorial, you learned how to generate text using KerasNLP and Gemma. Here are a few suggestions for what to learn next:\n",
"\n",
- "* Learn how to [finetune a Gemma model](https://ai.google.dev/gemma/docs/lora_tuning).\n",
- "* Learn how to perform [distributed fine-tuning and inference on a Gemma model](https://ai.google.dev/gemma/docs/distributed_tuning).\n",
+ "* Learn how to [finetune a Gemma model](https://ai.google.dev/gemma/docs/core/lora_tuning).\n",
+ "* Learn how to perform [distributed fine-tuning and inference on a Gemma model](https://ai.google.dev/gemma/docs/core/distributed_tuning).\n",
"* Learn about [Gemma integration with Vertex AI](https://ai.google.dev/gemma/docs/integrations/vertex)\n",
"* Learn how to [use Gemma models with Vertex AI](https://cloud.google.com/vertex-ai/docs/generative-ai/open-models/use-gemma)."
]
diff --git a/site/en/gemma/docs/lora_tuning.ipynb b/site/en/gemma/docs/core/lora_tuning.ipynb
similarity index 98%
rename from site/en/gemma/docs/lora_tuning.ipynb
rename to site/en/gemma/docs/core/lora_tuning.ipynb
index 2f6071358..ed8ea92f2 100644
--- a/site/en/gemma/docs/lora_tuning.ipynb
+++ b/site/en/gemma/docs/core/lora_tuning.ipynb
@@ -16,7 +16,7 @@
"id": "Tce3stUlHN0L"
},
"source": [
- "##### Copyright 2024 Google LLC."
+ "##### Copyright 2025 Google LLC."
]
},
{
@@ -58,15 +58,15 @@
"source": [
"
"
]
@@ -77,10 +77,6 @@
"id": "lSGRSsRPgkzK"
},
"source": [
- "## Overview\n",
- "\n",
- "Gemma is a family of lightweight, state-of-the art open models built from the same research and technology used to create the Gemini models.\n",
- "\n",
"Large Language Models (LLMs) like Gemma have been shown to be effective at a variety of NLP tasks. An LLM is first pre-trained on a large corpus of text in a self-supervised fashion. Pre-training helps LLMs learn general-purpose knowledge, such as statistical relationships between words. An LLM can then be fine-tuned with domain-specific data to perform downstream tasks (such as sentiment analysis).\n",
"\n",
"LLMs are extremely large in size (parameters in the order of billions). Full fine-tuning (which updates all the parameters in the model) is not required for most applications because typical fine-tuning datasets are relatively much smaller than the pre-training datasets.\n",
@@ -1000,7 +996,7 @@
"This tutorial covered LoRA fine-tuning on a Gemma model using KerasNLP. Check out the following docs next:\n",
"\n",
"* Learn how to [generate text with a Gemma model](https://ai.google.dev/gemma/docs/get_started).\n",
- "* Learn how to perform [distributed fine-tuning and inference on a Gemma model](https://ai.google.dev/gemma/docs/distributed_tuning).\n",
+ "* Learn how to perform [distributed fine-tuning and inference on a Gemma model](https://ai.google.dev/gemma/docs/core/distributed_tuning).\n",
"* Learn how to [use Gemma open models with Vertex AI](https://cloud.google.com/vertex-ai/docs/generative-ai/open-models/use-gemma).\n",
"* Learn how to [fine-tune Gemma using KerasNLP and deploy to Vertex AI](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/model_garden/model_garden_gemma_kerasnlp_to_vertexai.ipynb)."
]
diff --git a/site/en/gemma/docs/pytorch_gemma.ipynb b/site/en/gemma/docs/core/pytorch_gemma.ipynb
similarity index 66%
rename from site/en/gemma/docs/pytorch_gemma.ipynb
rename to site/en/gemma/docs/core/pytorch_gemma.ipynb
index 418c5ca3c..84d2c0e4c 100644
--- a/site/en/gemma/docs/pytorch_gemma.ipynb
+++ b/site/en/gemma/docs/core/pytorch_gemma.ipynb
@@ -16,7 +16,7 @@
"id": "Tce3stUlHN0L"
},
"source": [
- "##### Copyright 2024 Google LLC."
+ "##### Copyright 2025 Google LLC."
]
},
{
@@ -49,12 +49,12 @@
"source": [
"
![](\"https://ai.google.dev/static/site-assets/images/docs/notebook-site-button.png\")
View on ai.google.dev\n",
+ " ![](\"https://www.tensorflow.org/images/colab_logo_32px.png\"){kind=logo}
Run in Google Colab\n",
" ![](\"https://www.kaggle.com/static/images/logos/kaggle-logo-transparent-300.png\"){kind=spacer}
Run in Kaggle\n",
+ " ![](\"https://ai.google.dev/images/cloud-icon.svg\"){kind=icon}
Open in Vertex AI\n",
+ " ![](\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\")
View source on GitHub\n",
+ "