Skip to content

Commit 31d6194

Browse files
ricken07tzolov
ricken07
authored and
tzolov
committed
Add mistralai support documentations
 Also - fix a MistralAiApi (and OpenAiApi) issue just introduced by Mistral API - change the auto-config properties prefix from spring.ai.mistral.* to spring.ai.mistralai.* - minor doc references updates.
1 parent 076726c commit 31d6194

File tree

12 files changed

+428
-9
lines changed

12 files changed

+428
-9
lines changed

‎models/spring-ai-mistral-ai/src/main/java/org/springframework/ai/mistralai/api/MistralAiApi.java

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -586,7 +586,11 @@ public enum ChatCompletionFinishReason {
586586
/**
587587
* The content was omitted due to a flag from our content filters.
588588
*/
589-
@JsonProperty("model_length") MODEL_LENGTH
589+
@JsonProperty("model_length") MODEL_LENGTH,
590+
/**
591+
* The model called a tool.
592+
*/
593+
@JsonProperty("tool_call") TOOL_CALL
590594
// @formatter:on
591595

592596
}

‎models/spring-ai-openai/src/main/java/org/springframework/ai/openai/api/OpenAiApi.java

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -470,7 +470,11 @@ public enum ChatCompletionFinishReason {
470470
/**
471471
* (deprecated) The model called a function.
472472
*/
473-
@JsonProperty("function_call") FUNCTION_CALL
473+
@JsonProperty("function_call") FUNCTION_CALL,
474+
/**
475+
* Only for compatibility with Mistral AI API.
476+
*/
477+
@JsonProperty("tool_call") TOOL_CAL
474478
}
475479

476480
/**

‎spring-ai-docs/src/main/antora/modules/ROOT/nav.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
**** xref:api/embeddings/bedrock-cohere-embedding.adoc[Cohere]
1313
**** xref:api/embeddings/bedrock-titan-embedding.adoc[Titan]
1414
*** xref:api/embeddings/onnx.adoc[Transformers (ONNX)]
15+
*** xref:api/embeddings/mistralai-embeddings.adoc[Mistral AI]
1516
** xref:api/chatclient.adoc[]
1617
*** xref:api/clients/openai-chat.adoc[OpenAI]
1718
**** xref:api/clients/functions/openai-chat-functions.adoc[Function Calling]
@@ -28,6 +29,7 @@
2829
**** xref:api/clients/vertexai-palm2-chat.adoc[VertexAI PaLM2 ]
2930
**** xref:api/clients/vertexai-gemini-chat.adoc[VertexAI Gemini]
3031
***** xref:api/clients/functions/vertexai-gemini-chat-functions.adoc[Function Calling]
32+
*** xref:api/clients/mistralai-chat.adoc[Mistral AI]
3133
** xref:api/imageclient.adoc[]
3234
*** xref:api/clients/image/openai-image.adoc[OpenAI]
3335
*** xref:api/clients/image/stabilityai-image.adoc[Stability]

‎spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chatclient.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,7 @@ image::spring-ai-chat-completions-clients.jpg[align="center", width="800px"]
186186
** xref:api/clients/bedrock/bedrock-llama2.adoc[Llama2 Chat Completion]
187187
** xref:api/clients/bedrock/bedrock-titan.adoc[Titan Chat Completion]
188188
** xref:api/clients/bedrock/bedrock-anthropic.adoc[Anthropic Chat Completion]
189+
* xref:api/clients/mistralai-chat.adoc[Mistral AI Chat Completion] (streaming & function-calling support)
189190

190191
== Chat Model API
191192

‎spring-ai-docs/src/main/antora/modules/ROOT/pages/api/clients/mistralai-chat.adoc

Lines changed: 227 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,227 @@
1+
= Mistral AI Chat
2+
3+
Spring AI supports the various AI language models from Mistral AI. You can interact with Mistral AI language models and create a multilingual conversational assistant based on Mistral models.
4+
5+
== Prerequisites
6+
7+
You will need to create an API with MistralAI to access Mistral AI language models.
8+
Create an account at https://auth.mistral.ai/ui/registration[MistralAI registration page] and generate the token on the https://console.mistral.ai/api-keys/[API Keys page].
9+
The Spring AI project defines a configuration property named `spring.ai.mistralai.api-key` that you should set to the value of the `API Key` obtained from console.mistral.ai.
10+
Exporting an environment variable is one way to set that configuration property:
11+
12+
[source,shell]
13+
----
14+
export SPRING_AI_MISTRALAI_API_KEY=<INSERT KEY HERE>
15+
----
16+
17+
=== Add Repositories and BOM
18+
19+
Spring AI artifacts are published in Spring Milestone and Snapshot repositories.
20+
Refer to the xref:getting-started.adoc#repositories[Repositories] section to add these repositories to your build system.
21+
22+
To help with dependency management, Spring AI provides a BOM (bill of materials) to ensure that a consistent version of Spring AI is used throughout the entire project. Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build system.
23+
24+
25+
26+
== Auto-configuration
27+
28+
Spring AI provides Spring Boot auto-configuration for the MistralAI Chat Client.
29+
To enable it add the following dependency to your project's Maven `pom.xml` file:
30+
31+
[source, xml]
32+
----
33+
<dependency>
34+
<groupId>org.springframework.ai</groupId>
35+
<artifactId>spring-ai-mistral-ai-spring-boot-starter</artifactId>
36+
</dependency>
37+
----
38+
39+
or to your Gradle `build.gradle` build file.
40+
41+
[source,groovy]
42+
----
43+
dependencies {
44+
implementation 'org.springframework.ai:spring-ai-mistral-ai-spring-boot-starter'
45+
}
46+
----
47+
48+
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
49+
50+
=== Chat Properties
51+
52+
The prefix `spring.ai.mistralai` is used as the property prefix that lets you connect to OpenAI.
53+
54+
[cols="3,5,1"]
55+
|====
56+
| Property | Description | Default
57+
58+
| spring.ai.mistralai.base-url | The URL to connect to | https://api.mistral.ai
59+
| spring.ai.mistralai.api-key | The API Key | -
60+
|====
61+
62+
The prefix `spring.ai.mistralai.chat` is the property prefix that lets you configure the chat client implementation for MistralAI.
63+
64+
[cols="3,5,1"]
65+
|====
66+
| Property | Description | Default
67+
68+
| spring.ai.mistralai.chat.enabled | Enable MistralAI chat client. | true
69+
| spring.ai.mistralai.chat.base-url | Optional overrides the spring.ai.mistralai.base-url to provide chat specific url | -
70+
| spring.ai.mistralai.chat.api-key | Optional overrides the spring.ai.mistralai.api-key to provide chat specific api-key | -
71+
| spring.ai.mistralai.chat.options.model | This is the MistralAI Chat model to use | `mistral-tiny` (the `gpt-3.5-turbo`, `gpt-4`, and `gpt-4-32k` point to the latest model versions)
72+
| spring.ai.mistralai.chat.options.temperature | The sampling temperature to use that controls the apparent creativity of generated completions. Higher values will make output more random while lower values will make results more focused and deterministic. It is not recommended to modify temperature and top_p for the same completions request as the interaction of these two settings is difficult to predict. | 0.8
73+
| spring.ai.mistralai.chat.options.maxTokens | The maximum number of tokens to generate in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length. | -
74+
| spring.ai.mistralai.chat.options.safePrompt | Indicates whether to inject a security prompt before all conversations. | false
75+
| spring.ai.mistralai.chat.options.randomSeed | This feature is in Beta. If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. | -
76+
| spring.ai.mistralai.chat.options.stop | Up to 4 sequences where the API will stop generating further tokens. | -
77+
| spring.ai.mistralai.chat.options.topP | An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both. | -
78+
|====
79+
80+
NOTE: You can override the common `spring.ai.mistralai.base-url` and `spring.ai.mistralai.api-key` for the `ChatClient` and `EmbeddingClient` implementations.
81+
The `spring.ai.mistralai.chat.base-url` and `spring.ai.mistralai.chat.api-key` properties if set take precedence over the common properties.
82+
This is useful if you want to use different MistralAI accounts for different models and different model endpoints.
83+
84+
TIP: All properties prefixed with `spring.ai.mistralai.chat.options` can be overridden at runtime by adding a request specific <<chat-options>> to the `Prompt` call.
85+
86+
=== Chat Options [[chat-options]]
87+
88+
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistralai/src/main/java/org/springframework/ai/mistralai/MistralAiChatOptions.java[MistralAiChatOptions.java] provides model configurations, such as the model to use, the temperature, the frequency penalty, etc.
89+
90+
On start-up, the default options can be configured with the `MistralAiChatClient(api, options)` constructor or the `spring.ai.mistralai.chat.options.*` properties.
91+
92+
At run-time you can override the default options by adding new, request specific, options to the `Prompt` call.
93+
For example to override the default model and temperature for a specific request:
94+
95+
[source,java]
96+
----
97+
ChatResponse response = chatClient.call(
98+
new Prompt(
99+
"Generate the names of 5 famous pirates.",
100+
MistralAiChatOptions.builder()
101+
.withModel("mistral-medium")
102+
.withTemperature(0.5f)
103+
.build()
104+
));
105+
----
106+
107+
TIP: In addition to the model specific https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistralai/src/main/java/org/springframework/ai/mistralai/MistralAiChatOptions.java[MistralAiChatOptions] you can use a portable https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/ChatOptions.java[ChatOptions] instance, created with the https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/ChatOptionsBuilder.java[ChatOptionsBuilder#builder()].
108+
109+
=== Sample Controller (Auto-configuration)
110+
111+
https://start.spring.io/[Create] a new Spring Boot project and add the `spring-ai-mistralai-spring-boot-starter` to your pom (or gradle) dependencies.
112+
113+
Add a `application.properties` file, under the `src/main/resources` directory, to enable and configure the OpenAi Chat client:
114+
115+
[source,application.properties]
116+
----
117+
spring.ai.mistralai.api-key=YOUR_API_KEY
118+
spring.ai.mistralai.chat.options.model=mistral-medium
119+
spring.ai.mistralai.chat.options.temperature=0.7
120+
----
121+
122+
TIP: replace the `api-key` with your OpenAI credentials.
123+
124+
This will create a `MistralAiChatClient` implementation that you can inject into your class.
125+
Here is an example of a simple `@Controller` class that uses the chat client for text generations.
126+
127+
[source,java]
128+
----
129+
@RestController
130+
public class ChatController {
131+
132+
private final MistralAiChatClient chatClient;
133+
134+
@Autowired
135+
public ChatController(MistralAiChatClient chatClient) {
136+
this.chatClient = chatClient;
137+
}
138+
139+
@GetMapping("/ai/generate")
140+
public Map generate(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
141+
return Map.of("generation", chatClient.call(message));
142+
}
143+
144+
@GetMapping("/ai/generateStream")
145+
public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
146+
var prompt = new Prompt(new UserMessage(message));
147+
return chatClient.stream(prompt);
148+
}
149+
}
150+
----
151+
152+
== Manual Configuration
153+
154+
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistralai/src/main/java/org/springframework/ai/mistralai/MistralAiChatClient.java[MistralAiChatClient] implements the `ChatClient` and `StreamingChatClient` and uses the <<low-level-api>> to connect to the MistralAI service.
155+
156+
Add the `spring-ai-mistralai` dependency to your project's Maven `pom.xml` file:
157+
158+
[source, xml]
159+
----
160+
<dependency>
161+
<groupId>org.springframework.ai</groupId>
162+
<artifactId>spring-ai-mistralai</artifactId>
163+
</dependency>
164+
----
165+
166+
or to your Gradle `build.gradle` build file.
167+
168+
[source,groovy]
169+
----
170+
dependencies {
171+
implementation 'org.springframework.ai:spring-ai-mistralai'
172+
}
173+
----
174+
175+
TIP: Refer to the xref:getting-started.adoc#dependency-management[Dependency Management] section to add the Spring AI BOM to your build file.
176+
177+
Next, create a `MistralAiChatClient` and use it for text generations:
178+
179+
[source,java]
180+
----
181+
var mistralAiApi = new MistralAiApi(System.getenv("MISTRAL_AI_API_KEY"));
182+
183+
var chatClient = new MistralAiChatClient(mistralAiApi, MistralAiChatOptions.builder()
184+
.withModel("mistral-small")
185+
.withTemperature(0.4f)
186+
.withMaxToken(200)
187+
.build());
188+
189+
ChatResponse response = chatClient.call(
190+
new Prompt("Generate the names of 5 famous pirates."));
191+
192+
// Or with streaming responses
193+
Flux<ChatResponse> response = chatClient.stream(
194+
new Prompt("Generate the names of 5 famous pirates."));
195+
----
196+
197+
The `MistralAiChatOptions` provides the configuration information for the chat requests.
198+
The `MistralAiChatOptions.Builder` is fluent options builder.
199+
200+
=== Low-level MistralAiApi Client [[low-level-api]]
201+
202+
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistralai/src/main/java/org/springframework/ai/mistralai/api/MistralAiApi.java[MistralAiApi] provides is lightweight Java client for link:https://docs.mistral.ai/api/[Mistral AI API].
203+
204+
Here is a simple snippet how to use the api programmatically:
205+
206+
[source,java]
207+
----
208+
MistralAiApi mistralAiApi =
209+
new MistralAiApi(System.getenv("MISTRAL_AI_API_KEY"));
210+
211+
ChatCompletionMessage chatCompletionMessage =
212+
new ChatCompletionMessage("Hello world", Role.USER);
213+
214+
// Sync request
215+
ResponseEntity<ChatCompletion> response = mistralAiApi.chatCompletionEntity(
216+
new ChatCompletionRequest(List.of(chatCompletionMessage), "mistral-small", 0.8f, false));
217+
218+
// Streaming request
219+
Flux<ChatCompletionChunk> streamResponse = mistralAiApi.chatCompletionStream(
220+
new ChatCompletionRequest(List.of(chatCompletionMessage), "mistral-small", 0.8f, true));
221+
----
222+
223+
Follow the https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistral-ai/src/main/java/org/springframework/ai/mistralai/api/MistralAiApi.java[MistralAiApi.java]'s JavaDoc for further information.
224+
225+
==== MistralAiApi Samples
226+
The link:https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-mistral-ai/src/test/java/org/springframework/ai/mistralai/api/MistralAiApiIT.java[MistralAiApiIT.java] test provides some general examples how to use the lightweight library.
227+

‎spring-ai-docs/src/main/antora/modules/ROOT/pages/api/embeddings.adoc

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -157,4 +157,5 @@ Internally the various `EmbeddingClient` implementations use different low-level
157157
* xref:api/embeddings/postgresml-embeddings.adoc[Spring AI PostgresML Embeddings]
158158
* xref:api/embeddings/bedrock-cohere-embedding.adoc[Spring AI Bedrock Cohere Embeddings]
159159
* xref:api/embeddings/bedrock-titan-embedding.adoc[Spring AI Bedrock Titan Embeddings]
160-
* xref:api/embeddings/vertexai-embeddings.adoc[Spring AI VertexAI Embeddings]
160+
* xref:api/embeddings/vertexai-embeddings.adoc[Spring AI VertexAI PaLM2 Embeddings]
161+
* xref:api/embeddings/mistralai-embeddings.adoc[Spring AI Mistral AI Embeddings]

0 commit comments

Comments
 (0)