Skip to content

Commit 683a1b9

Browse files
tzolovtzolov
committed
Clarify bedrock anthropic and llama2 docs
1 parent ced9e6a commit 683a1b9

File tree

2 files changed

+154
-153
lines changed

2 files changed

+154
-153
lines changed

‎spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/bedrock/bedrock-anthropic.adoc

Lines changed: 82 additions & 80 deletions
Original file line numberDiff line numberDiff line change
@@ -10,62 +10,11 @@ The Claude model has the following high level features
1010
1111
The https://aws.amazon.com/bedrock/claude[AWS Bedrock Anthropic Model Page] and https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html[Amazon Bedrock User Guide] contains detailed information on how to use the AWS hosted model.
1212

13-
== Getting Started
13+
== Pre-requisites
1414

1515
Refer to the xref:api/clients/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
1616

17-
The link:./src/main/java/org/springframework/ai/bedrock/anthropic/BedrockAnthropicChatClient.java[BedrockAnthropicChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the `AnthropicChatBedrockApi` library to connect to the Bedrock Anthropic service.
18-
19-
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
20-
21-
[source,xml]
22-
----
23-
<dependency>
24-
<groupId>org.springframework.ai</groupId>
25-
<artifactId>spring-ai-bedrock</artifactId>
26-
<version>0.8.0-SNAPSHOT</version>
27-
</dependency>
28-
----
29-
30-
or to your Gradle `build.gradle` build file.
31-
32-
[source,gradle]
33-
----
34-
dependencies {
35-
implementation 'org.springframework.ai:spring-ai-bedrock:0.8.0-SNAPSHOT'
36-
}
37-
----
38-
39-
NOTE: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
40-
41-
Next, create an `BedrockAnthropicChatClient` instance and use it to text generations requests:
42-
43-
[source,java]
44-
----
45-
AnthropicChatBedrockApi anthropicApi = new AnthropicChatBedrockApi(
46-
AnthropicChatBedrockApi.AnthropicModel.CLAUDE_V2.id(),
47-
EnvironmentVariableCredentialsProvider.create(),
48-
Region.EU_CENTRAL_1.id(),
49-
new ObjectMapper());
50-
51-
BedrockAnthropicChatClient chatClient = new BedrockAnthropicChatClient(anthropicApi,
52-
AnthropicChatOptions.builder()
53-
.withTemperature(0.6f)
54-
.withTopK(10)
55-
.withTopP(0.8f)
56-
.withMaxTokensToSample(100)
57-
.withAnthropicVersion(AnthropicChatBedrockApi.DEFAULT_ANTHROPIC_VERSION)
58-
.build());
59-
60-
ChatResponse response = chatClient.call(
61-
new Prompt("Generate the names of 5 famous pirates."));
62-
63-
// Or with streaming responses
64-
Flux<ChatResponse> response = chatClient.stream(
65-
new Prompt("Generate the names of 5 famous pirates."));
66-
----
67-
68-
=== AnthropicChatClient Auto-configuration
17+
== Auto-configuration
6918

7019
or you can leverage the `spring-ai-bedrock-ai-spring-boot-starter` Spring Boot starter:
7120

@@ -78,18 +27,50 @@ or you can leverage the `spring-ai-bedrock-ai-spring-boot-starter` Spring Boot s
7827
</dependency>
7928
----
8029

81-
==== Enable Anthropic Support
30+
=== Enable Anthropic Support
8231

8332
Spring AI defines a configuration property named `spring.ai.bedrock.anthropic.chat.enabled` that you should set to `true` to enable support for Anthropic.
84-
8533
Exporting environment variables in one way to set this configuration property.
8634

8735
[source,shell]
8836
----
8937
export SPRING_AI_BEDROCK_ANTHROPIC_CHAT_ENABLED=true
9038
----
9139

92-
==== Sample Code
40+
=== Chat Properties
41+
42+
The prefix `spring.ai.bedrock.aws` is the property prefix to configure the connection to AWS Bedrock.
43+
44+
[cols="3,3,1"]
45+
|====
46+
| Property | Description | Default
47+
48+
| spring.ai.bedrock.aws.region | AWS region to use. | us-east-1
49+
| spring.ai.bedrock.aws.access-key | AWS access key. | -
50+
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
51+
|====
52+
53+
The prefix `spring.ai.bedrock.anthropic.chat` is the property prefix that configures the `ChatClient` implementation for Claude.
54+
55+
[cols="2,5,1"]
56+
|====
57+
| Property | Description | Default
58+
59+
| spring.ai.bedrock.anthropic.chat.enable | Enable Bedrock Anthropic chat client. Disabled by default | false
60+
| spring.ai.bedrock.anthropic.chat.model | The model id to use. See the `AnthropicChatModel` for the supported models. | anthropic.claude-v2
61+
| spring.ai.bedrock.anthropic.chat.options.temperature | Controls the randomness of the output. Values can range over [0.0,1.0] | 0.8
62+
| spring.ai.bedrock.anthropic.chat.options.topP | The maximum cumulative probability of tokens to consider when sampling. | AWS Bedrock default
63+
| spring.ai.bedrock.anthropic.chat.options.topK | Specify the number of token choices the generative uses to generate the next token. | AWS Bedrock default
64+
| spring.ai.bedrock.anthropic.chat.options.stopSequences | Configure up to four sequences that the generative recognizes. After a stop sequence, the generative stops generating further tokens. The returned text doesn't contain the stop sequence. | 10
65+
| spring.ai.bedrock.anthropic.chat.options.anthropicVersion | The version of the generative to use. | bedrock-2023-05-31
66+
| spring.ai.bedrock.anthropic.chat.options.maxTokensToSample | Specify the maximum number of tokens to use in the generated response. Note that the models may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate. We recommend a limit of 4,000 tokens for optimal performance. | 500
67+
|====
68+
69+
Look at the Spring AI enumeration `AnthropicChatModel` for other model IDs. The other value supported is `anthropic.claude-instant-v1`.
70+
71+
Model ID values can also be found in the https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html[AWS Bedrock documentation for base model IDs].
72+
73+
=== Sample Code
9374

9475
This will create a `ChatClient` implementation that you can inject into your class.
9576

@@ -122,38 +103,59 @@ public class ChatController {
122103
}
123104
----
124105

125-
==== Bedrock Properties
106+
== Manual Configuration
126107

127-
The prefix `spring.ai.bedrock.aws` is the property prefix to configure the connection to AWS Bedrock.
108+
The link:./src/main/java/org/springframework/ai/bedrock/anthropic/BedrockAnthropicChatClient.java[BedrockAnthropicChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the `AnthropicChatBedrockApi` library to connect to the Bedrock Anthropic service.
128109

129-
[cols="3,3,1"]
130-
|====
131-
| Property | Description | Default
110+
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
132111

133-
| spring.ai.bedrock.aws.region | AWS region to use. | us-east-1
134-
| spring.ai.bedrock.aws.access-key | AWS access key. | -
135-
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
136-
|====
112+
[source,xml]
113+
----
114+
<dependency>
115+
<groupId>org.springframework.ai</groupId>
116+
<artifactId>spring-ai-bedrock</artifactId>
117+
<version>0.8.0-SNAPSHOT</version>
118+
</dependency>
119+
----
137120

138-
The prefix `spring.ai.bedrock.anthropic.chat` is the property prefix that configures the `ChatClient` implementation for Claude.
121+
or to your Gradle `build.gradle` build file.
139122

140-
[cols="2,5,1"]
141-
|====
142-
| Property | Description | Default
123+
[source,gradle]
124+
----
125+
dependencies {
126+
implementation 'org.springframework.ai:spring-ai-bedrock:0.8.0-SNAPSHOT'
127+
}
128+
----
143129

144-
| spring.ai.bedrock.anthropic.chat.enable | Enable Bedrock Anthropic chat client. Disabled by default | false
145-
| spring.ai.bedrock.anthropic.chat.model | The model id to use. See the `AnthropicChatModel` for the supported models. | anthropic.claude-v2
146-
| spring.ai.bedrock.anthropic.chat.options.temperature | Controls the randomness of the output. Values can range over [0.0,1.0] | 0.8
147-
| spring.ai.bedrock.anthropic.chat.options.topP | The maximum cumulative probability of tokens to consider when sampling. | AWS Bedrock default
148-
| spring.ai.bedrock.anthropic.chat.options.topK | Specify the number of token choices the generative uses to generate the next token. | AWS Bedrock default
149-
| spring.ai.bedrock.anthropic.chat.options.stopSequences | Configure up to four sequences that the generative recognizes. After a stop sequence, the generative stops generating further tokens. The returned text doesn't contain the stop sequence. | 10
150-
| spring.ai.bedrock.anthropic.chat.options.anthropicVersion | The version of the generative to use. | bedrock-2023-05-31
151-
| spring.ai.bedrock.anthropic.chat.options.maxTokensToSample | Specify the maximum number of tokens to use in the generated response. Note that the models may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate. We recommend a limit of 4,000 tokens for optimal performance. | 500
152-
|====
130+
NOTE: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
153131

154-
Look at the Spring AI enumeration `AnthropicChatModel` for other model IDs. The other value supported is `anthropic.claude-instant-v1`.
132+
Next, create an `BedrockAnthropicChatClient` instance and use it to text generations requests:
133+
134+
[source,java]
135+
----
136+
AnthropicChatBedrockApi anthropicApi = new AnthropicChatBedrockApi(
137+
AnthropicChatBedrockApi.AnthropicModel.CLAUDE_V2.id(),
138+
EnvironmentVariableCredentialsProvider.create(),
139+
Region.EU_CENTRAL_1.id(),
140+
new ObjectMapper());
141+
142+
BedrockAnthropicChatClient chatClient = new BedrockAnthropicChatClient(anthropicApi,
143+
AnthropicChatOptions.builder()
144+
.withTemperature(0.6f)
145+
.withTopK(10)
146+
.withTopP(0.8f)
147+
.withMaxTokensToSample(100)
148+
.withAnthropicVersion(AnthropicChatBedrockApi.DEFAULT_ANTHROPIC_VERSION)
149+
.build());
150+
151+
ChatResponse response = chatClient.call(
152+
new Prompt("Generate the names of 5 famous pirates."));
153+
154+
// Or with streaming responses
155+
Flux<ChatResponse> response = chatClient.stream(
156+
new Prompt("Generate the names of 5 famous pirates."));
157+
----
155158

156-
Model ID values can also be found in the https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html[AWS Bedrock documentation for base model IDs].
157159

158160
== Appendices
159161

‎spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/bedrock/bedrock-llama2.adoc

Lines changed: 72 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -9,58 +9,11 @@ Rigorous testing, including over 1,000 hours of red-teaming and annotation, ensu
99

1010
The https://aws.amazon.com/bedrock/llama-2/[AWS Llama 2 Model Page] and https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html[Amazon Bedrock User Guide] contains detailed information on how to use the AWS hosted model.
1111

12-
13-
== Getting Started
12+
== Prerequisites
1413

1514
Refer to the xref:api/clients/bedrock.adoc[Spring AI documentation on Amazon Bedrock] for setting up API access.
1615

17-
18-
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
19-
20-
[source,xml]
21-
----
22-
<dependency>
23-
<groupId>org.springframework.ai</groupId>
24-
<artifactId>spring-ai-bedrock</artifactId>
25-
<version>0.8.0-SNAPSHOT</version>
26-
</dependency>
27-
----
28-
29-
or to your Gradle `build.gradle` build file.
30-
31-
[source,gradle]
32-
----
33-
dependencies {
34-
implementation 'org.springframework.ai:spring-ai-bedrock:0.8.0-SNAPSHOT'
35-
}
36-
----
37-
38-
NOTE: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
39-
40-
The link:./src/main/java/org/springframework/ai/bedrock/llama2/BedrockLlama2ChatClient.java[BedrockLlama2ChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the `Llama2ChatBedrockApi` library to connect to the Bedrock Llama2 service.
41-
42-
Here is how to create and use a `BedrockLlama2ChatClient`:
43-
44-
[source,java]
45-
----
46-
Llama2ChatBedrockApi api = new Llama2ChatBedrockApi(Llama2ChatModel.LLAMA2_70B_CHAT_V1.id(),
47-
EnvironmentVariableCredentialsProvider.create(), Region.US_EAST_1.id(), new ObjectMapper());
48-
49-
BedrockLlama2ChatClient chatClient = new BedrockLlama2ChatClient(api,
50-
BedrockLlama2ChatOptions.builder()
51-
.withTemperature(0.5f)
52-
.withMaxGenLen(100)
53-
.withTopP(0.9f).build());
54-
55-
ChatResponse response = chatClient.call(
56-
new Prompt("Generate the names of 5 famous pirates."));
57-
58-
// Or with streaming responses
59-
Flux<ChatResponse> response = chatClient.stream(
60-
new Prompt("Generate the names of 5 famous pirates."));
61-
----
62-
63-
=== BedrockLlama2ChatClient Auto-configuration
16+
== Auto-configuration
6417

6518
or you can leverage the `spring-ai-bedrock-ai-spring-boot-starter` Spring Boot starter:
6619

@@ -82,18 +35,48 @@ dependencies {
8235
}
8336
----
8437

85-
==== Enable Llama2 Chat Support
38+
=== Enable Llama2 Chat Support
8639

8740
Spring AI defines a configuration property named `spring.ai.bedrock.llama2.chat.enabled` that you should set to `true` to enable support for Llama2.
88-
8941
Exporting environment variables in one way to set this configuration property.
9042

9143
[source,shell]
9244
----
9345
export SPRING_AI_BEDROCK_LLAMA2_CHAT_ENABLED=true
9446
----
9547

96-
==== Sample Code
48+
=== Chat Properties
49+
50+
The prefix `spring.ai.bedrock.aws` is the property prefix to configure the connection to AWS Bedrock.
51+
52+
[cols="3,3,3"]
53+
|====
54+
| Property | Description | Default
55+
56+
| spring.ai.bedrock.aws.region | AWS region to use. | us-east-1
57+
| spring.ai.bedrock.aws.access-key | AWS access key. | -
58+
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
59+
|====
60+
61+
62+
The prefix `spring.ai.bedrock.llama2.chat` is the property prefix that configures the `ChatClient` implementation for Llama2.
63+
64+
[cols="2,5,1"]
65+
|====
66+
| Property | Description | Default
67+
68+
| spring.ai.bedrock.llama2.chat.enabled | Enable or disable support for Llama2 | false
69+
| spring.ai.bedrock.llama2.chat.model | The model id to use (See Below) | meta.llama2-70b-chat-v1
70+
| spring.ai.bedrock.llama2.chat.options.temperature | Controls the randomness of the output. Values can range over [0.0,1.0], inclusive. A value closer to 1.0 will produce responses that are more varied, while a value closer to 0.0 will typically result in less surprising responses from the model. This value specifies default to be used by the backend while making the call to the model. | 0.7
71+
| spring.ai.bedrock.llama2.chat.options.top-p | The maximum cumulative probability of tokens to consider when sampling. The model uses combined Top-k and nucleus sampling. Nucleus sampling considers the smallest set of tokens whose probability sum is at least topP. | AWS Bedrock default
72+
| spring.ai.bedrock.llama2.chat.options.max-gen-len | Specify the maximum number of tokens to use in the generated response. The model truncates the response once the generated text exceeds maxGenLen. | 300
73+
|====
74+
75+
Look at the Spring AI enumeration, `Llama2ChatModel` for other model IDs. The other value supported is `meta.llama2-13b-chat-v1`.
76+
77+
Model ID values can also be found in the https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html[AWS Bedrock documentation for base model IDs].
78+
79+
=== Sample Code
9780

9881
This will create a `ChatClient` implementation that you can inject into your class.
9982

@@ -126,36 +109,52 @@ public class ChatController {
126109
}
127110
----
128111

129-
=== Bedrock Properties
112+
== Manual Configuration
130113

131-
The prefix `spring.ai.bedrock.aws` is the property prefix to configure the connection to AWS Bedrock.
114+
Add the `spring-ai-bedrock` dependency to your project's Maven `pom.xml` file:
132115

133-
[cols="3,3,3"]
134-
|====
135-
| Property | Description | Default
116+
[source,xml]
117+
----
118+
<dependency>
119+
<groupId>org.springframework.ai</groupId>
120+
<artifactId>spring-ai-bedrock</artifactId>
121+
<version>0.8.0-SNAPSHOT</version>
122+
</dependency>
123+
----
136124

137-
| spring.ai.bedrock.aws.region | AWS region to use. | us-east-1
138-
| spring.ai.bedrock.aws.access-key | AWS access key. | -
139-
| spring.ai.bedrock.aws.secret-key | AWS secret key. | -
140-
|====
125+
or to your Gradle `build.gradle` build file.
141126

127+
[source,gradle]
128+
----
129+
dependencies {
130+
implementation 'org.springframework.ai:spring-ai-bedrock:0.8.0-SNAPSHOT'
131+
}
132+
----
142133

143-
The prefix `spring.ai.bedrock.llama2.chat` is the property prefix that configures the `ChatClient` implementation for Llama2.
134+
NOTE: Refer to the xref:getting-started.adoc#_dependency_management[Dependency Management] section to add Milestone and/or Snapshot Repositories to your build file.
144135

145-
[cols="2,5,1"]
146-
|====
147-
| Property | Description | Default
136+
The link:./src/main/java/org/springframework/ai/bedrock/llama2/BedrockLlama2ChatClient.java[BedrockLlama2ChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the `Llama2ChatBedrockApi` library to connect to the Bedrock Llama2 service.
148137

149-
| spring.ai.bedrock.llama2.chat.enabled | Enable or disable support for Llama2 | false
150-
| spring.ai.bedrock.llama2.chat.model | The model id to use (See Below) | meta.llama2-70b-chat-v1
151-
| spring.ai.bedrock.llama2.chat.options.temperature | Controls the randomness of the output. Values can range over [0.0,1.0], inclusive. A value closer to 1.0 will produce responses that are more varied, while a value closer to 0.0 will typically result in less surprising responses from the model. This value specifies default to be used by the backend while making the call to the model. | 0.7
152-
| spring.ai.bedrock.llama2.chat.options.top-p | The maximum cumulative probability of tokens to consider when sampling. The model uses combined Top-k and nucleus sampling. Nucleus sampling considers the smallest set of tokens whose probability sum is at least topP. | AWS Bedrock default
153-
| spring.ai.bedrock.llama2.chat.options.max-gen-len | Specify the maximum number of tokens to use in the generated response. The model truncates the response once the generated text exceeds maxGenLen. | 300
154-
|====
138+
Here is how to create and use a `BedrockLlama2ChatClient`:
155139

156-
Look at the Spring AI enumeration, `Llama2ChatModel` for other model IDs. The other value supported is `meta.llama2-13b-chat-v1`.
140+
[source,java]
141+
----
142+
Llama2ChatBedrockApi api = new Llama2ChatBedrockApi(Llama2ChatModel.LLAMA2_70B_CHAT_V1.id(),
143+
EnvironmentVariableCredentialsProvider.create(), Region.US_EAST_1.id(), new ObjectMapper());
157144
158-
Model ID values can also be found in the https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html[AWS Bedrock documentation for base model IDs].
145+
BedrockLlama2ChatClient chatClient = new BedrockLlama2ChatClient(api,
146+
BedrockLlama2ChatOptions.builder()
147+
.withTemperature(0.5f)
148+
.withMaxGenLen(100)
149+
.withTopP(0.9f).build());
150+
151+
ChatResponse response = chatClient.call(
152+
new Prompt("Generate the names of 5 famous pirates."));
153+
154+
// Or with streaming responses
155+
Flux<ChatResponse> response = chatClient.stream(
156+
new Prompt("Generate the names of 5 famous pirates."));
157+
----
159158

160159

161160
== Appendices

0 commit comments

Comments
 (0)