Skip to content

Commit a49e9ac

Browse files
msampathkumarmsampathkumar
authored
docs: Generative ai docs (#12720)
* docs(generative-ai): update README.md Updated the page to provide overview of the feature folders. * feat(generative-ai): add an example template folder
1 parent ffbe8ed commit a49e9ac

File tree

7 files changed

+368
-84
lines changed

7 files changed

+368
-84
lines changed

‎generative_ai/README.md

Lines changed: 175 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -1,84 +1,175 @@
1-
# Generative AI on Google Cloud
2-
3-
* Product Page: https://cloud.google.com/ai/generative-ai?hl=en
4-
* Code samples: https://cloud.google.com/docs/samples?text=Generative%20AI
5-
6-
## Samples Style Guide
7-
8-
If you are interested in code sample contributions, see [Contributing Guide](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/CONTRIBUTING.md).
9-
10-
Only for Generative AI Python samples, below style guide instructions take precedence over [Samples Style Guide](https://googlecloudplatform.github.io/samples-style-guide/).
11-
12-
### 1. Use Scripting format
13-
14-
Wrap the imports, sample code and the region tags to be with in one function definition.
15-
This is to keep the region tag section code to be in script format and also allowing you to write regular testcases.
16-
17-
> This change is motivated by the desire to provide code samples in a copy-paste-run
18-
format that is helpful for popular data science community tools like
19-
Google Colab, Jupyter Notebooks, and IPython shell.
20-
21-
Here is an example code.
22-
23-
```python
24-
def create_hello_world_file(filename):
25-
# <region tag: starts here>
26-
import os
27-
28-
# TODO(developer): Update and uncomment below code
29-
# filename = `/tmp/test.txt`
30-
31-
if os.path.isfile(filename):
32-
print(f'Overriding content in file(name: {filename})!')
33-
34-
with open(filename) as fp:
35-
fp.write('Hello world!')
36-
# <region tag: ends here>
37-
```
38-
39-
In Google Cloud documentation, this code sample will be shown as below
40-
41-
```python
42-
import os
43-
44-
# TODO(developer): Update and uncomment below code
45-
# filename = `/tmp/test.txt`
46-
47-
if os.path.isfile(filename):
48-
print(f'Overriding content in file(name: {filename})!')
49-
50-
with open(filename) as fp:
51-
fp.write('Hello world!')
52-
```
53-
54-
**Note:** In the above sample, `imports` are include and `TODO's` are provided to variable
55-
that need to update by users.
56-
57-
### 2. Avoid Hidden Variables
58-
59-
Suggestion to avoid hidden variables in code samples
60-
61-
* Use global variables, to defined common variables like PROJECT_ID, LOCATION.
62-
63-
* Keep the function definitions simple, with less or no arguments.
64-
* Ex: Use `def generate_text() -> str:` instead of `def generate_text(temperature=..image_path=..video_path=..) -> str:`
65-
66-
* Use descriptive variables names and if need, use long variable name.
67-
* Ex: Use `text_input` instead of `user_input` or `input_var` or `myvar` ...
68-
69-
* Resist the temptation to tell more
70-
* Ex: Don't define unused optional arguments
71-
* Ex: Use `Read more @ http://..` than explaining `video_config looks like {"foo": "bar",...}`
72-
73-
**Note**: Not all the samples are the same! "Avoid Hidden variables" is not same as "Dont/No Hidden Variables".
74-
75-
76-
## Conclusion
77-
78-
To summarize, it's crucial to maintain the simplicity and brevity of your code
79-
examples.
80-
> The ideal sample is one that appears self-evident and immediately
81-
comprehensible.
82-
83-
84-
1+
# Generative AI Samples on Google Cloud
2+
3+
Welcome to the Python samples folder for Generative AI on Vertex AI! In this folder, you can find the Python samples
4+
used in [Google Cloud Generative AI documentation](https://cloud.google.com/ai/generative-ai?hl=en).
5+
6+
If you are looking for colab notebook, then this [link](https://github.com/GoogleCloudPlatform/generative-ai/tree/main).
7+
8+
## Getting Started
9+
10+
To try and run these Code samples, we have following recommend using Google Cloud IDE or Google Colab.
11+
12+
Note: A Google Cloud Project is a pre-requisite.
13+
14+
### Feature folders
15+
16+
Browse the folders below to find the Generative AI capabilities you're interested in.
17+
18+
<table>
19+
<tr>
20+
<td><strong>Python Samples Folder</strong>
21+
</td>
22+
<td><strong>Google Cloud Product</strong>
23+
</td>
24+
<td><strong>Short Description (With the help of Gemini 1.5)</strong>
25+
</td>
26+
</tr>
27+
<tr>
28+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/context_caching">Context Caching</a>
29+
</td>
30+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-overview">https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-overview</a>
31+
</td>
32+
<td>Code samples demonstrating how to use context caching with Vertex AI's generative models. This allows for more consistent and relevant responses across multiple interactions by storing previous conversation history.
33+
</td>
34+
</tr>
35+
<tr>
36+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/controlled_generation">Controlled Generation</a>
37+
</td>
38+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/control-generated-output">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/control-generated-output</a>
39+
</td>
40+
<td>Examples of how to control the output of generative models, such as specifying length, format, or sentiment.
41+
</td>
42+
</tr>
43+
<tr>
44+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/count_token">Count Token</a>
45+
</td>
46+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/list-token">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/list-token</a>
47+
</td>
48+
<td>Code demonstrating how to count tokens in text, which is crucial for managing costs and understanding model limitations.
49+
</td>
50+
</tr>
51+
<tr>
52+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/embeddings">Embeddings</a>
53+
</td>
54+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings">https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings</a>
55+
</td>
56+
<td>Code showing how to generate and use embeddings from text or images. Embeddings can be used for tasks like semantic search, clustering, and classification.
57+
</td>
58+
</tr>
59+
<tr>
60+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/extensions">Extensions</a>
61+
</td>
62+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/extensions/overview">https://cloud.google.com/vertex-ai/generative-ai/docs/extensions/overview</a>
63+
</td>
64+
<td>Demonstrations of how to use extensions with generative models, enabling them to access and process real-time information, use tools, and interact with external systems.
65+
</td>
66+
</tr>
67+
<tr>
68+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/function_calling">Function Calling</a>
69+
</td>
70+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling</a>
71+
</td>
72+
<td>Examples of how to use function calling to enable generative models to execute specific actions or retrieve information from external APIs.
73+
</td>
74+
</tr>
75+
<tr>
76+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/grounding">Grounding</a>
77+
</td>
78+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/overview">https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/overview</a>
79+
</td>
80+
<td>Code illustrating how to ground generative models with specific knowledge bases or data sources to improve the accuracy and relevance of their responses.
81+
</td>
82+
</tr>
83+
<tr>
84+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/image_generation">Image Generation</a>
85+
</td>
86+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview">https://cloud.google.com/vertex-ai/generative-ai/docs/image/overview</a>
87+
</td>
88+
<td>Samples showcasing how to generate images from text prompts using models like Imagen.
89+
</td>
90+
</tr>
91+
<tr>
92+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/model_garden">Model Garden</a>
93+
</td>
94+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models">https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models</a>
95+
</td>
96+
<td>Resources related to exploring and utilizing pre-trained models available in Vertex AI's Model Garden.
97+
</td>
98+
</tr>
99+
<tr>
100+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/model_tuning">Model Tuning</a>
101+
</td>
102+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/models/tune-models">https://cloud.google.com/vertex-ai/generative-ai/docs/models/tune-models</a>
103+
</td>
104+
<td>Code and guides for fine-tuning pre-trained generative models on specific datasets or for specific tasks.
105+
</td>
106+
</tr>
107+
<tr>
108+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/rag">RAG</a>
109+
</td>
110+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/rag-api">https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/rag-api</a>
111+
</td>
112+
<td>Information and resources about Retrieval Augmented Generation (RAG), which combines information retrieval with generative models.
113+
</td>
114+
</tr>
115+
<tr>
116+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/reasoning_engine">Reasoning Engine</a>
117+
</td>
118+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/reasoning-engine">https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/reasoning-engine</a>
119+
</td>
120+
<td>Details about the Reasoning Engine, which enables more complex reasoning and logical deduction in generative models.
121+
</td>
122+
</tr>
123+
<tr>
124+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/safety">Safety</a>
125+
</td>
126+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-attributes">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/configure-safety-attributes</a>
127+
</td>
128+
<td>Examples of how to configure safety attributes and filters to mitigate risks and ensure responsible use of generative models.
129+
</td>
130+
</tr>
131+
<tr>
132+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/system_instructions">System Instructions</a>
133+
</td>
134+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/system-instructions?hl=en">https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/system-instructions?hl=en</a>
135+
</td>
136+
<td>Code demonstrating how to provide system instructions to guide the behavior and responses of generative models.
137+
</td>
138+
</tr>
139+
<tr>
140+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/text_generation">Text Generation</a>
141+
</td>
142+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/send-chat-prompts-gemini">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/send-chat-prompts-gemini</a>
143+
</td>
144+
<td>Samples of how to generate text using Gemini models, including chat-based interactions and creative writing.
145+
</td>
146+
</tr>
147+
<tr>
148+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/audio">Understand Audio</a>
149+
</td>
150+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/audio-understanding">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/audio-understanding</a>
151+
</td>
152+
<td>Examples of how to use generative models for audio understanding tasks, such as transcription and audio classification.
153+
</td>
154+
</tr>
155+
<tr>
156+
<td><a href="https://github.com/GoogleCloudPlatform/python-docs-samples/tree/main/generative_ai/video">Understand Video</a>
157+
</td>
158+
<td><a href="https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/video-understanding">https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/video-understanding</a>
159+
</td>
160+
<td>Samples showcasing how to use generative models for video understanding tasks, such as video summarization and content analysis.
161+
</td>
162+
</tr>
163+
</table>
164+
165+
## Contributing
166+
167+
Contributions welcome! See the [Contributing Guide](https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/CONTRIBUTING.md).
168+
169+
## Getting help
170+
171+
Please use the [issues page](https://github.com/GoogleCloudPlatform/python-docs-samples/issues) to provide suggestions, feedback or submit a bug report.
172+
173+
## Disclaimer
174+
175+
This repository itself is not an officially supported Google product. The code in this repository is for demonstrative purposes only.

‎generative_ai/template_folder/advanced_example.py

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# # Copyright 2024 Google LLC
2+
# #
3+
# # Licensed under the Apache License, Version 2.0 (the "License");
4+
# # you may not use this file except in compliance with the License.
5+
# # You may obtain a copy of the License at
6+
# #
7+
# # https://www.apache.org/licenses/LICENSE-2.0
8+
# #
9+
# # Unless required by applicable law or agreed to in writing, software
10+
# # distributed under the License is distributed on an "AS IS" BASIS,
11+
# # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# # See the License for the specific language governing permissions and
13+
# # limitations under the License.
14+
# import os
15+
#
16+
# from vertexai.generative_models import GenerationResponse
17+
#
18+
# PROJECT_ID = os.getenv("GOOGLE_CLOUD_PROJECT")
19+
#
20+
#
21+
# def advanced_example() -> GenerationResponse:
22+
# # [START generativeaionvertexai_gemini_token_count_multimodal]
23+
# import vertexai
24+
# from vertexai.generative_models import GenerativeModel, Part
25+
#
26+
# # TODO(developer): Update and un-comment below line
27+
# # PROJECT_ID = "your-project-id"
28+
# vertexai.init(project=PROJECT_ID, location="us-central1")
29+
#
30+
# model = GenerativeModel("gemini-1.5-flash-002")
31+
#
32+
# contents = [
33+
# Part.from_uri(
34+
# "gs://cloud-samples-data/generative-ai/video/pixel8.mp4",
35+
# mime_type="video/mp4",
36+
# ),
37+
# "Provide a description of the video.",
38+
# ]
39+
#
40+
# # tokens count for user prompt
41+
# response = model.count_tokens(contents)
42+
# print(f"Prompt Token Count: {response.total_tokens}")
43+
# print(f"Prompt Character Count: {response.total_billable_characters}")
44+
# # Example response:
45+
# # Prompt Token Count: 16822
46+
# # Prompt Character Count: 30
47+
#
48+
# # Send text to Gemini
49+
# response = model.generate_content(contents)
50+
# usage_metadata = response.usage_metadata
51+
#
52+
# # tokens count for model response
53+
# print(f"Prompt Token Count: {usage_metadata.prompt_token_count}")
54+
# print(f"Candidates Token Count: {usage_metadata.candidates_token_count}")
55+
# print(f"Total Token Count: {usage_metadata.total_token_count}")
56+
# # Example response:
57+
# # Prompt Token Count: 16822
58+
# # Candidates Token Count: 71
59+
# # Total Token Count: 16893
60+
#
61+
# # [END generativeaionvertexai_gemini_token_count_multimodal]
62+
# return response
63+
#
64+
#
65+
# if __name__ == "__main__":
66+
# advanced_example()

‎generative_ai/template_folder/noxfile_config.py

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Copyright 2021 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
# Default TEST_CONFIG_OVERRIDE for python repos.
16+
17+
# You can copy this file into your directory, then it will be imported from
18+
# the noxfile.py.
19+
20+
# The source of truth:
21+
# https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/noxfile_config.py
22+
23+
TEST_CONFIG_OVERRIDE = {
24+
# You can opt out from the test for specific Python versions.
25+
"ignored_versions": ["2.7", "3.7", "3.8", "3.10", "3.11"],
26+
# Old samples are opted out of enforcing Python type hints
27+
# All new samples should feature them
28+
"enforce_type_hints": True,
29+
# An envvar key for determining the project id to use. Change it
30+
# to 'BUILD_SPECIFIC_GCLOUD_PROJECT' if you want to opt in using a
31+
# build specific Cloud project. You can also use your own string
32+
# to use your own Cloud project.
33+
"gcloud_project_env": "GOOGLE_CLOUD_PROJECT",
34+
# 'gcloud_project_env': 'BUILD_SPECIFIC_GCLOUD_PROJECT',
35+
# If you need to use a specific version of pip,
36+
# change pip_version_override to the string representation
37+
# of the version number, for example, "20.2.4"
38+
"pip_version_override": None,
39+
# A dictionary you want to inject into your test. Don't put any
40+
# secrets here. These values will override predefined values.
41+
"envs": {},
42+
}

‎generative_ai/template_folder/requirements-test.txt

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
backoff==2.2.1
2+
google-api-core==2.19.0
3+
pytest==8.2.0
4+
pytest-asyncio==0.23.6

‎generative_ai/template_folder/requirements.txt

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
pandas==1.3.5; python_version == '3.7'
2+
pandas==2.0.3; python_version == '3.8'
3+
pandas==2.1.4; python_version > '3.8'
4+
pillow==10.3.0; python_version < '3.8'
5+
pillow==10.3.0; python_version >= '3.8'
6+
google-cloud-aiplatform[all]==1.69.0
7+
sentencepiece==0.2.0
8+
google-auth==2.29.0
9+
anthropic[vertex]==0.28.0
10+
langchain-core==0.2.11
11+
langchain-google-vertexai==1.0.6
12+
numpy<2
13+
openai==1.30.5
14+
immutabledict==4.2.0

0 commit comments

Comments
 (0)