Skip to content

Commit 056b95a

Browse files
tzolovtzolov
committed
Improve Chat Completon API documentation
 - minor code style improvements.
1 parent 7fca784 commit 056b95a

File tree

16 files changed

+139
-489
lines changed

16 files changed

+139
-489
lines changed

‎models/spring-ai-bedrock/README_ANTHROPIC_CHAT.md

Lines changed: 1 addition & 119 deletions
Original file line numberDiff line numberDiff line change
@@ -2,122 +2,4 @@
22

33
Provides Bedrock Anthropic Chat API and Spring-AI chat clients.
44

5-
## BedrockAnthropicChatClient
6-
7-
The [BedrockAnthropicChatClient](./src/main/java/org/springframework/ai/bedrock/anthropic/BedrockAnthropicChatClient.java) implements the `ChatClient` and `StreamingChatClient` and uses the `AnthropicChatBedrockApi` library to connect to the Bedrock Anthropic service.
8-
9-
Add the `spring-ai-` dependency to your project's Maven `pom.xml` file:
10-
11-
```xml
12-
<dependency>
13-
<groupId>org.springframework.ai</groupId>
14-
<artifactId>spring-ai-bedrock</artifactId>
15-
<version>0.8.0-SNAPSHOT</version>
16-
</dependency>
17-
```
18-
19-
or to your Gradle `build.gradle` build file.
20-
21-
```gradle
22-
dependencies {
23-
implementation 'org.springframework.ai:spring-ai-bedrock:0.8.0-SNAPSHOT'
24-
}
25-
```
26-
27-
Next, create an `BedrockAnthropicChatClient` instance and use it to text generations requests:
28-
29-
```java
30-
AnthropicChatBedrockApi anthropicApi = new AnthropicChatBedrockApi(
31-
AnthropicChatBedrockApi.AnthropicModel.CLAUDE_V2.id(),
32-
EnvironmentVariableCredentialsProvider.create(),
33-
Region.EU_CENTRAL_1.id(),
34-
new ObjectMapper());
35-
36-
BedrockAnthropicChatClient chatClient = new BedrockAnthropicChatClient(anthropicApi,
37-
AnthropicChatOptions.builder()
38-
.withTemperature(0.6f)
39-
.withTopK(10)
40-
.withTopP(0.8f)
41-
.withMaxTokensToSample(100)
42-
.withAnthropicVersion(AnthropicChatBedrockApi.DEFAULT_ANTHROPIC_VERSION)
43-
.build());
44-
45-
ChatResponse response = chatClient.call(
46-
new Prompt("Generate the names of 5 famous pirates."));
47-
48-
// Or with streaming responses
49-
Flux<ChatResponse> response = chatClient.stream(
50-
new Prompt("Generate the names of 5 famous pirates."));
51-
```
52-
53-
or you can leverage the `spring-ai-bedrock-ai-spring-boot-starter` Spring Boot starter:
54-
55-
```xml
56-
<dependency>
57-
<groupId>org.springframework.ai</groupId>
58-
<artifactId>spring-ai-bedrock-ai-spring-boot-starter</artifactId>
59-
<version>0.8.0-SNAPSHOT</version>
60-
</dependency>
61-
```
62-
63-
And set `spring.ai.bedrock.anthropic.chat.enabled=true`.
64-
By default the client is disabled.
65-
66-
Use the `BedrockAnthropicChatProperties` to configure the Bedrock Anthropic Chat client:
67-
68-
| Property | Description | Default |
69-
| ------------- | ------------- | ------------- |
70-
| spring.ai.bedrock.aws.region | AWS region to use. | us-east-1 |
71-
| spring.ai.bedrock.aws.accessKey | AWS credentials access key. | |
72-
| spring.ai.bedrock.aws.secretKey | AWS credentials secret key. | |
73-
| spring.ai.bedrock.anthropic.chat.enable | Enable Bedrock Anthropic chat client. Disabled by default | false |
74-
| spring.ai.bedrock.anthropic.chat.model | The model id to use. See the `AnthropicChatModel` for the supported models. | anthropic.claude-v2 |
75-
| spring.ai.bedrock.anthropic.chat.options.temperature | Controls the randomness of the output. Values can range over [0.0,1.0] | 0.8 |
76-
| spring.ai.bedrock.anthropic.chat.options.topP | The maximum cumulative probability of tokens to consider when sampling. | AWS Bedrock default |
77-
| spring.ai.bedrock.anthropic.chat.options.topK | Specify the number of token choices the generative uses to generate the next token. | AWS Bedrock default |
78-
| spring.ai.bedrock.anthropic.chat.options.stopSequences | Configure up to four sequences that the generative recognizes. After a stop sequence, the generative stops generating further tokens. The returned text doesn't contain the stop sequence. | 10 |
79-
| spring.ai.bedrock.anthropic.chat.options.anthropicVersion | The version of the generative to use. | bedrock-2023-05-31 |
80-
| spring.ai.bedrock.anthropic.chat.options.maxTokensToSample | Specify the maximum number of tokens to use in the generated response. Note that the models may stop before reaching this maximum. This parameter only specifies the absolute maximum number of tokens to generate. We recommend a limit of 4,000 tokens for optimal performance. | 500 |
81-
82-
## Appendices
83-
84-
## Using low-level AnthropicChatBedrockApi Library
85-
86-
[AnthropicChatBedrockApi](./src/main/java/org/springframework/ai/bedrock/anthropic/api/AnthropicChatBedrockApi.java) provides is lightweight Java client on top of AWS Bedrock [Anthropic Claude models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-claude.html).
87-
88-
Following class diagram illustrates the AnthropicChatBedrockApi interface and building blocks:
89-
90-
![AnthropicChatBedrockApi Class Diagram](./src/test/resources/doc/Bedrock-Anthropic-Chat-API.jpg)
91-
92-
The AnthropicChatBedrockApi supports the `anthropic.claude-instant-v1` and `anthropic.claude-v2` models.
93-
94-
Also the AnthropicChatBedrockApi supports both synchronous (e.g. `chatCompletion()`) and streaming (e.g. `chatCompletionStream()`) responses.
95-
96-
Here is a simple snippet how to use the api programmatically:
97-
98-
```java
99-
AnthropicChatBedrockApi anthropicChatApi = new AnthropicChatBedrockApi(
100-
AnthropicModel.CLAUDE_V2.id(),
101-
Region.EU_CENTRAL_1.id());
102-
103-
AnthropicChatRequest request = AnthropicChatRequest
104-
.builder(String.format(AnthropicChatBedrockApi.PROMPT_TEMPLATE, "Name 3 famous pirates"))
105-
.withTemperature(0.8f)
106-
.withMaxTokensToSample(300)
107-
.withTopK(10)
108-
// .withStopSequences(List.of("\n\nHuman:"))
109-
.build();
110-
111-
AnthropicChatResponse response = anthropicChatApi.chatCompletion(request);
112-
113-
System.out.println(response.completion());
114-
115-
// Streaming response
116-
Flux<AnthropicChatResponse> responseStream = anthropicChatApi.chatCompletionStream(request);
117-
118-
List<AnthropicChatResponse> responses = responseStream.collectList().block();
119-
120-
System.out.println(responses);
121-
```
122-
123-
Follow the [AnthropicChatBedrockApi.java](./src/main/java/org/springframework/ai/bedrock/anthropic/api/AnthropicChatBedrockApi.java)'s JavaDoc for further information.
5+
Visit the Spring AI [Bedrock Anthropic Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/bedrock/bedrock-anthropic.html).

‎models/spring-ai-bedrock/README_COHERE_CHAT.md

Lines changed: 2 additions & 107 deletions
Original file line numberDiff line numberDiff line change
@@ -1,110 +1,5 @@
1-
# 1. Bedrock Cohere Chat
1+
# Bedrock Cohere Chat
22

33
Provides Bedrock Cohere Chat clients.
44

5-
## 1.1 CohereChatBedrockApi
6-
7-
[CohereChatBedrockApi](./src/main/java/org/springframework/ai/bedrock/cohere/api/CohereChatBedrockApi.java) provides is lightweight Java client on top of AWS Bedrock [Cohere Command models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-cohere-command.html).
8-
9-
Following class diagram illustrates the Llama2ChatBedrockApi interface and building blocks:
10-
11-
![CohereChatBedrockApi Class Diagram](./src/test/resources/doc/Bedrock%20Cohere%20Chat%20API.jpg)
12-
13-
The CohereChatBedrockApi supports the `cohere.command-light-text-v14` and `cohere.command-text-v14` models for bot synchronous (e.g. `chatCompletion()`) and streaming (e.g. `chatCompletionStream()`) responses.
14-
15-
Here is a simple snippet how to use the api programmatically:
16-
17-
```java
18-
CohereChatBedrockApi cohereChatApi = new CohereChatBedrockApi(
19-
CohereChatModel.COHERE_COMMAND_V14.id(),
20-
Region.US_EAST_1.id());
21-
22-
var request = CohereChatRequest
23-
.builder("What is the capital of Bulgaria and what is the size? What it the national anthem?")
24-
.withStream(false)
25-
.withTemperature(0.5f)
26-
.withTopP(0.8f)
27-
.withTopK(15)
28-
.withMaxTokens(100)
29-
.withStopSequences(List.of("END"))
30-
.withReturnLikelihoods(CohereChatRequest.ReturnLikelihoods.ALL)
31-
.withNumGenerations(3)
32-
.withLogitBias(null)
33-
.withTruncate(Truncate.NONE)
34-
.build();
35-
36-
CohereChatResponse response = cohereChatApi.chatCompletion(request);
37-
38-
var request = CohereChatRequest
39-
.builder("What is the capital of Bulgaria and what is the size? What it the national anthem?")
40-
.withStream(true)
41-
.withTemperature(0.5f)
42-
.withTopP(0.8f)
43-
.withTopK(15)
44-
.withMaxTokens(100)
45-
.withStopSequences(List.of("END"))
46-
.withReturnLikelihoods(CohereChatRequest.ReturnLikelihoods.ALL)
47-
.withNumGenerations(3)
48-
.withLogitBias(null)
49-
.withTruncate(Truncate.NONE)
50-
.build();
51-
52-
Flux<CohereChatResponse.Generation> responseStream = cohereChatApi.chatCompletionStream(request);
53-
List<CohereChatResponse.Generation> responses = responseStream.collectList().block();
54-
```
55-
56-
## 1.2 BedrockCohereChatClient
57-
58-
[BedrockCohereChatClient](./src/main/java/org/springframework/ai/bedrock/cohere/BedrockCohereChatClient.java) implements the Spring-Ai `ChatClient` and `StreamingChatClient` on top of the `CohereChatBedrockApi`.
59-
60-
You can use like this:
61-
62-
```java
63-
@Bean
64-
public CohereChatBedrockApi cohereApi() {
65-
return new CohereChatBedrockApi(
66-
CohereChatModel.COHERE_COMMAND_V14.id(),
67-
EnvironmentVariableCredentialsProvider.create(),
68-
Region.US_EAST_1.id(),
69-
new ObjectMapper());
70-
}
71-
72-
@Bean
73-
public BedrockCohereChatClient cohereChatClient(CohereChatBedrockApi cohereApi) {
74-
return new BedrockCohereChatClient(cohereApi);
75-
}
76-
```
77-
78-
or you can leverage the `spring-ai-bedrock-ai-spring-boot-starter` Boot starter. For this add the following dependency:
79-
80-
```xml
81-
<dependency>
82-
<artifactId>spring-ai-bedrock-ai-spring-boot-starter</artifactId>
83-
<groupId>org.springframework.ai</groupId>
84-
<version>0.8.0-SNAPSHOT</version>
85-
</dependency>
86-
```
87-
88-
**NOTE:** You have to enable the Bedrock Cohere chat client with `spring.ai.bedrock.cohere.chat.enabled=true`.
89-
By default the client is disabled.
90-
91-
Use the `BedrockCohereChatProperties` to configure the Bedrock Cohere Chat client:
92-
93-
| Property | Description | Default |
94-
| ------------- | ------------- | ------------- |
95-
| spring.ai.bedrock.aws.region | AWS region to use. | us-east-1 |
96-
| spring.ai.bedrock.aws.accessKey | AWS credentials access key. | |
97-
| spring.ai.bedrock.aws.secretKey | AWS credentials secret key. | |
98-
| spring.ai.bedrock.cohere.chat.enable | Enable Bedrock Cohere chat client. Disabled by default | false |
99-
| spring.ai.bedrock.cohere.chat.model | The model id to use. See the `CohereChatModel` for the supported models. | cohere.command-text-v14 |
100-
| spring.ai.bedrock.cohere.chat.temperature | Controls the randomness of the output. Values can range over [0.0,1.0] | 0.7 |
101-
| spring.ai.bedrock.cohere.chat.topP | The maximum cumulative probability of tokens to consider when sampling. | AWS Bedrock default |
102-
| spring.ai.bedrock.cohere.chat.topK | Specify the number of token choices the model uses to generate the next token | AWS Bedrock default |
103-
| spring.ai.bedrock.cohere.chat.maxTokens | Specify the maximum number of tokens to use in the generated response. | AWS Bedrock default |
104-
| spring.ai.bedrock.cohere.chat.stopSequences | Configure up to four sequences that the model recognizes. | AWS Bedrock default |
105-
| spring.ai.bedrock.cohere.chat.returnLikelihoods | The token likelihoods are returned with the response. | AWS Bedrock default |
106-
| spring.ai.bedrock.cohere.chat.numGenerations | The maximum number of generations that the model should return. | AWS Bedrock default |
107-
| spring.ai.bedrock.cohere.chat.logitBiasToken | Prevents the model from generating unwanted tokens or incentivize the model to include desired tokens. | AWS Bedrock default |
108-
| spring.ai.bedrock.cohere.chat.logitBiasBias | Prevents the model from generating unwanted tokens or incentivize the model to include desired tokens. | AWS Bedrock default |
109-
| spring.ai.bedrock.cohere.chat.truncate | Specifies how the API handles inputs longer than the maximum token length | AWS Bedrock default |
110-
5+
Visit the Spring AI [Bedrock Cohere Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/bedrock/bedrock-cohere.html).

‎models/spring-ai-bedrock/README_LLAMA2_CHAT.md

Lines changed: 2 additions & 88 deletions
Original file line numberDiff line numberDiff line change
@@ -1,91 +1,5 @@
1-
# 1. Bedrock Llama2
1+
# Bedrock Llama2
22

33
Provides Bedrock Llama2 Chat API and Spring-AI chat clients.
44

5-
## 1.1 Llama2ChatBedrockApi
6-
7-
[Llama2ChatBedrockApi](./src/main/java/org/springframework/ai/bedrock/llama2/api/Llama2ChatBedrockApi.java) provides is lightweight Java client on top of AWS Bedrock [Meta Llama 2 and Llama 2 Chat models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-meta.html).
8-
9-
Following class diagram illustrates the Llama2ChatBedrockApi interface and building blocks:
10-
11-
![Llama2ChatBedrockApi Class Diagram](./src/test/resources/doc/Bedrock-Llama2-Chat-API.jpg)
12-
13-
The Llama2ChatBedrockApi supports the `meta.llama2-13b-chat-v1` and `meta.llama2-70b-chat-v1` models.
14-
15-
Also the Llama2ChatBedrockApi supports both synchronous (e.g. `chatCompletion()`) and streaming (e.g. `chatCompletionStream()`) responses.
16-
17-
Here is a simple snippet how to use the api programmatically:
18-
19-
```java
20-
Llama2ChatBedrockApi llama2ChatApi = new Llama2ChatBedrockApi(
21-
Llama2ChatModel.LLAMA2_70B_CHAT_V1.id(),
22-
Region.US_EAST_1.id());
23-
24-
Llama2ChatRequest request = Llama2ChatRequest.builder("Hello, my name is")
25-
.withTemperature(0.9f)
26-
.withTopP(0.9f)
27-
.withMaxGenLen(20)
28-
.build();
29-
30-
Llama2ChatResponse response = llama2ChatApi.chatCompletion(request);
31-
32-
System.out.println(response.generation());
33-
34-
// Streaming response
35-
Flux<Llama2ChatResponse> responseStream = llama2ChatApi.chatCompletionStream(request);
36-
37-
List<Llama2ChatResponse> responses = responseStream.collectList().block();
38-
39-
System.out.println(responses);
40-
```
41-
42-
Follow the [Llama2ChatBedrockApi.java](./src/main/java/org/springframework/ai/bedrock/llama2/api/Llama2ChatBedrockApi.java)'s JavaDoc for further information.
43-
44-
## 1.2 BedrockLlama2ChatClient
45-
46-
[BedrockLlama2ChatClient](./src/main/java/org/springframework/ai/bedrock/llama2/BedrockLlama2ChatClient.java) implements the Spring-Ai `ChatClient` and `StreamingChatClient` on top of the `Llama2ChatBedrockApi`.
47-
48-
You can use like this:
49-
50-
```java
51-
@Bean
52-
public Llama2ChatBedrockApi llama2Api() {
53-
return new Llama2ChatBedrockApi(
54-
Llama2ChatModel.LLAMA2_70B_CHAT_V1.id(),
55-
EnvironmentVariableCredentialsProvider.create(),
56-
Region.US_EAST_1.id(),
57-
new ObjectMapper());
58-
}
59-
60-
@Bean
61-
public BedrockLlama2ChatClient llama2ChatClient(Llama2ChatBedrockApi llama2Api) {
62-
return new BedrockLlama2ChatClient(llama2Api);
63-
}
64-
```
65-
66-
or you can leverage the `spring-ai-bedrock-ai-spring-boot-starter` Spring Boot starter:
67-
68-
```xml
69-
<dependency>
70-
<artifactId>spring-ai-bedrock-ai-spring-boot-starter</artifactId>
71-
<groupId>org.springframework.ai</groupId>
72-
<version>0.8.0-SNAPSHOT</version>
73-
</dependency>
74-
```
75-
76-
And set `spring.ai.bedrock.llama2.chat.enabled=true`.
77-
By default the client is disabled.
78-
79-
Use the `BedrockLlama2ChatProperties` to configure the Bedrock Llama2 Chat client:
80-
81-
| Property | Description | Default |
82-
| ------------- | ------------- | ------------- |
83-
| spring.ai.bedrock.aws.region | AWS region to use. | us-east-1 |
84-
| spring.ai.bedrock.aws.accessKey | AWS credentials access key. | |
85-
| spring.ai.bedrock.aws.secretKey | AWS credentials secret key. | |
86-
| spring.ai.bedrock.llama2.chat.enable | Enable Bedrock Llama2 chat client. Disabled by default | false |
87-
| spring.ai.bedrock.llama2.chat.temperature | Controls the randomness of the output. Values can range over [0.0,1.0] | 0.7 |
88-
| spring.ai.bedrock.llama2.chat.topP | The maximum cumulative probability of tokens to consider when sampling. | AWS Bedrock default |
89-
| spring.ai.bedrock.llama2.chat.maxGenLen | Specify the maximum number of tokens to use in the generated response. | 300 |
90-
| spring.ai.bedrock.llama2.chat.model | The model id to use. See the `Llama2ChatModel` for the supported models. | meta.llama2-70b-chat-v1 |
91-
5+
Visit the Spring AI [Bedrock Llama2 Chat Documentation](https://docs.spring.io/spring-ai/reference/api/clients/bedrock/bedrock-llama2.html).

‎models/spring-ai-bedrock/src/main/java/org/springframework/ai/bedrock/llama2/BedrockLlama2ChatClient.java

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -18,19 +18,17 @@
1818

1919
import java.util.List;
2020

21-
import org.springframework.ai.chat.ChatResponse;
2221
import reactor.core.publisher.Flux;
2322

2423
import org.springframework.ai.bedrock.MessageToPromptConverter;
25-
import org.springframework.ai.bedrock.anthropic.AnthropicChatOptions;
26-
import org.springframework.ai.bedrock.anthropic.api.AnthropicChatBedrockApi.AnthropicChatRequest;
2724
import org.springframework.ai.bedrock.llama2.api.Llama2ChatBedrockApi;
2825
import org.springframework.ai.bedrock.llama2.api.Llama2ChatBedrockApi.Llama2ChatRequest;
2926
import org.springframework.ai.bedrock.llama2.api.Llama2ChatBedrockApi.Llama2ChatResponse;
3027
import org.springframework.ai.chat.ChatClient;
3128
import org.springframework.ai.chat.ChatOptions;
32-
import org.springframework.ai.chat.StreamingChatClient;
29+
import org.springframework.ai.chat.ChatResponse;
3330
import org.springframework.ai.chat.Generation;
31+
import org.springframework.ai.chat.StreamingChatClient;
3432
import org.springframework.ai.chat.metadata.ChatGenerationMetadata;
3533
import org.springframework.ai.chat.metadata.Usage;
3634
import org.springframework.ai.chat.prompt.Prompt;

‎spring-ai-core/src/main/java/org/springframework/ai/chat/ChatClient.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ default String call(String message) {
2828
return call(prompt).getResult().getOutput().getContent();
2929
}
3030

31+
@Override
3132
ChatResponse call(Prompt prompt);
3233

3334
}

‎spring-ai-core/src/main/java/org/springframework/ai/chat/ChatResponse.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ public Generation getResult() {
7777
* @return Returns {@link ChatResponseMetadata} containing information about the use
7878
* of the AI provider's API.
7979
*/
80+
@Override
8081
public ChatResponseMetadata getMetadata() {
8182
return this.chatResponseMetadata;
8283
}

‎spring-ai-core/src/main/java/org/springframework/ai/chat/Generation.java

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@ public AssistantMessage getOutput() {
4646
return this.assistantMessage;
4747
}
4848

49+
@Override
4950
public ChatGenerationMetadata getMetadata() {
5051
ChatGenerationMetadata chatGenerationMetadata = this.chatGenerationMetadata;
5152
return chatGenerationMetadata != null ? chatGenerationMetadata : ChatGenerationMetadata.NULL;

‎spring-ai-core/src/main/java/org/springframework/ai/chat/StreamingChatClient.java

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,15 @@
1616

1717
package org.springframework.ai.chat;
1818

19+
import reactor.core.publisher.Flux;
20+
1921
import org.springframework.ai.chat.prompt.Prompt;
2022
import org.springframework.ai.model.StreamingModelClient;
2123

2224
@FunctionalInterface
2325
public interface StreamingChatClient extends StreamingModelClient<Prompt, ChatResponse> {
2426

27+
@Override
28+
Flux<ChatResponse> stream(Prompt prompt);
29+
2530
}

0 commit comments

Comments
 (0)