Skip to content

Commit 2da161c

Browse files
committed
Improve OpenAI structured outputs support
- Add `TEXT` to the supported ResponseFormat types - Add JsonSchema record for structured output configuration - Update OpenAI chat documentation with details on Structured Outputs - Clarify usage of JSON_SCHEMA response format in properties and code examples - Update Structured Output Converter docs to mention OpenAI Structured Outputs
1 parent 91afed5 commit 2da161c

File tree

3 files changed

+44
-24
lines changed

3 files changed

+44
-24
lines changed

models/spring-ai-openai/src/main/java/org/springframework/ai/openai/api/OpenAiApi.java

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -519,13 +519,21 @@ public static Object FUNCTION(String functionName) {
519519
/**
520520
* An object specifying the format that the model must output.
521521
* @param type Must be one of 'text' or 'json_object'.
522+
* @param jsonSchema JSON schema object that describes the format of the JSON object.
523+
* Only applicable when type is 'json_schema'.
522524
*/
523525
@JsonInclude(Include.NON_NULL)
524526
public record ResponseFormat(
525527
@JsonProperty("type") Type type,
526528
@JsonProperty("json_schema") JsonSchema jsonSchema ) {
527529

528530
public enum Type {
531+
/**
532+
* Generates a text response. (default)
533+
*/
534+
@JsonProperty("text")
535+
TEXT,
536+
529537
/**
530538
* Enables JSON mode, which guarantees the message
531539
* the model generates is valid JSON.
@@ -541,6 +549,13 @@ public enum Type {
541549
JSON_SCHEMA
542550
}
543551

552+
/**
553+
* JSON schema object that describes the format of the JSON object.
554+
* Applicable for the 'json_schema' type only.
555+
* @param name The name of the schema.
556+
* @param schema The JSON schema object that describes the format of the JSON object.
557+
* @param strict If true, the model will only generate outputs that match the schema.
558+
*/
544559
@JsonInclude(Include.NON_NULL)
545560
public record JsonSchema(
546561
@JsonProperty("name") String name,
@@ -552,16 +567,16 @@ public JsonSchema(String name, String schema) {
552567
}
553568

554569
public JsonSchema(String name, String schema, Boolean strict) {
555-
this(StringUtils.hasText(name)? name : "custom_response_format_schema", ModelOptionsUtils.jsonToMap(schema), strict);
570+
this(StringUtils.hasText(name)? name : "custom_schema", ModelOptionsUtils.jsonToMap(schema), strict);
556571
}
557572
}
558573

559574
public ResponseFormat(Type type) {
560575
this(type, (JsonSchema) null);
561576
}
562577

563-
public ResponseFormat(Type type, String jsonSchena) {
564-
this(type, "custom_response_format_schema", jsonSchena, true);
578+
public ResponseFormat(Type type, String schema) {
579+
this(type, "custom_schema", schema, true);
565580
}
566581

567582
@ConstructorBinding

spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chat/openai-chat.adoc

Lines changed: 25 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -106,9 +106,9 @@ The prefix `spring.ai.openai.chat` is the property prefix that lets you configur
106106
| spring.ai.openai.chat.options.presencePenalty | Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. | -
107107
| spring.ai.openai.chat.options.responseFormat.type | Compatible with `GPT-4o`, `GPT-4o mini`, `GPT-4 Turbo` and all `GPT-3.5 Turbo` models newer than `gpt-3.5-turbo-1106`. The `JSON_OBJECT` type enables JSON mode, which guarantees the message the model generates is valid JSON.
108108
The `JSON_SCHEMA` type enables link:https://platform.openai.com/docs/guides/structured-outputs[Structured Outputs] which guarantees the model will match your supplied JSON schema. The JSON_SCHEMA type requires setting the `responseFormat.schema` property as well. | -
109-
| spring.ai.openai.chat.options.responseFormat.name | Reponse format schema name. Applicable only for `responseFormat.type=JSON_SCHEMA` | custom_response_format_schema
110-
| spring.ai.openai.chat.options.responseFormat.schema | Reponse format JSON schema. Applicable only for `responseFormat.type=JSON_SCHEMA` | -
111-
| spring.ai.openai.chat.options.responseFormat.strict | Reponse format JSON schema adheranse strictens. Applicable only for `responseFormat.type=JSON_SCHEMA` | -
109+
| spring.ai.openai.chat.options.responseFormat.name | Response format schema name. Applicable only for `responseFormat.type=JSON_SCHEMA` | custom_schema
110+
| spring.ai.openai.chat.options.responseFormat.schema | Response format JSON schema. Applicable only for `responseFormat.type=JSON_SCHEMA` | -
111+
| spring.ai.openai.chat.options.responseFormat.strict | Response format JSON schema adherence strictness. Applicable only for `responseFormat.type=JSON_SCHEMA` | -
112112
| spring.ai.openai.chat.options.seed | This feature is in Beta. If specified, our system will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. | -
113113
| spring.ai.openai.chat.options.stop | Up to 4 sequences where the API will stop generating further tokens. | -
114114
| spring.ai.openai.chat.options.topP | An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. We generally recommend altering this or temperature but not both. | -
@@ -210,22 +210,24 @@ for carrying. The bowl is placed on a flat surface with a neutral-colored backgr
210210
view of the fruit inside.
211211
----
212212

213-
== Structured Output
213+
== Structured Outputs
214214

215-
In addition to existing, model agnostic, xref::api/structured-output-converter.adoc[Structured Output Converter] utilities,
216-
OpenAi provides custom https://platform.openai.com/docs/guides/structured-outputs[Structured Outputs] API that guarantees
217-
the model will always generate responses that adhere to your supplied https://json-schema.org/overview/what-is-jsonschema[JSON Schema].
215+
OpenAI provides custom https://platform.openai.com/docs/guides/structured-outputs[Structured Outputs] APIs that ensure your model generates responses conforming strictly to your provided `JSON Schema`.
216+
In addition to the existing Spring AI model-agnostic xref::api/structured-output-converter.adoc[Structured Output Converter], these APIs offer enhanced control and precision.
218217

219-
Spring AI offers flexible options to configure your response-format either manually using the OpenAiChatOptions builder or using application properties.
218+
NOTE: Currently, OpenAI supports a link:https://platform.openai.com/docs/guides/structured-outputs/supported-schemas[subset of the JSON Schema language] format.
220219

221-
=== with chat options builder
220+
=== Configuration
222221

223-
The OpenAiChatOptions builder allow you to set the response format programmatically like this:
222+
Spring AI allows you to configure your response format either programmatically using the `OpenAiChatOptions` builder or through application properties.
223+
224+
==== Using the Chat Options Builder
225+
226+
You can set the response format programmatically with the `OpenAiChatOptions` builder as shown below:
224227

225228
[source,java]
226229
----
227-
228-
var jsonSchema = """
230+
String jsonSchema = """
229231
{
230232
"type": "object",
231233
"properties": {
@@ -255,11 +257,13 @@ Prompt prompt = new Prompt("how can I solve 8x + 7 = -23",
255257
.build());
256258
257259
ChatResponse response = this.openAiChatModel.call(prompt);
258-
259260
----
260261

261-
You can combine this with the existing xref::api/structured-output-converter.adoc#_bean_output_converter[BeanOutputConverter] utilities to
262-
generate the JSON schema from your domain objects and convert the response into domain instances:
262+
NOTE: Adhere to the OpenAI link:https://platform.openai.com/docs/guides/structured-outputs/supported-schemas[subset of the JSON Schema language] format.
263+
264+
==== Integrating with BeanOutputConverter Utilities
265+
266+
You can leverage existing xref::api/structured-output-converter.adoc#_bean_output_converter[BeanOutputConverter] utilities to automatically generate the JSON Schema from your domain objects and later convert the structured response into domain-specific instances:
263267

264268
[source,java]
265269
----
@@ -293,12 +297,13 @@ String content = response.getResult().getOutput().getContent();
293297
MathReasoning mathReasoning = outputConverter.convert(content);
294298
----
295299

296-
NOTE: You must use the `@JsonProperty(required = true,..)` annotation to ensure the generated schema produces the `required[...]` field.
297-
Although optional for JSON Schema, OpenAI requires this filed for the structured response to work.
300+
NOTE: Ensure you use the `@JsonProperty(required = true,...)` annotation.
301+
This is crucial for generating a schema that accurately marks fields as `required`.
302+
Although this is optional for JSON Schema, OpenAI link:https://platform.openai.com/docs/guides/structured-outputs/all-fields-must-be-required[mandates] it for the structured response to function correctly.
298303

299-
=== with application properties
304+
==== Configuring via Application Properties
300305

301-
You can use the `spring.ai.openai.chat.options.response-format.*` application properties to configure your desired response format.
306+
Alternatively, when using the OpenAI auto-configuration, you can configure the desired response format through the following application properties:
302307

303308
[source,application.properties]
304309
----
@@ -307,7 +312,7 @@ spring.ai.openai.chat.options.model=gpt-4o-mini
307312
308313
spring.ai.openai.chat.options.response-format.type=JSON_SCHEMA
309314
spring.ai.openai.chat.options.response-format.name=MySchemaName
310-
spring.ai.openai.chat.options.response-format.schema=`{"type":"object","properties":{"steps":{"type":"array","items":{"type":"object","properties":{"explanation":{"type":"string"},"output":{"type":"string"}},"required":["explanation","output"],"additionalProperties":false}},"final_answer":{"type":"string"}},"required":["steps","final_answer"],"additionalProperties":false}`
315+
spring.ai.openai.chat.options.response-format.schema={"type":"object","properties":{"steps":{"type":"array","items":{"type":"object","properties":{"explanation":{"type":"string"},"output":{"type":"string"}},"required":["explanation","output"],"additionalProperties":false}},"final_answer":{"type":"string"}},"required":["steps","final_answer"],"additionalProperties":false}
311316
spring.ai.openai.chat.options.response-format.strict=true
312317
313318
----

spring-ai-docs/src/main/antora/modules/ROOT/pages/api/structured-output-converter.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -262,7 +262,7 @@ The following AI Models have been tested to support List, Map and Bean structure
262262

263263
Some AI Models provide dedicated configuration options to generate structured (usually JSON) output.
264264

265-
* xref:api/chat/openai-chat.adoc[OpenAI] - provides a `spring.ai.openai.chat.options.responseFormat` options specifying the format that the model must output. Setting to `{ "type": "json_object" }` enables JSON mode, which guarantees the message the model generates is valid JSON.
265+
* xref:api/chat/openai-chat.adoc#_structured_outputs[OpenAI Structured Outputs] can ensure your model generates responses conforming strictly to your provided JSON Schema. You can choose between the `JSON_OBJECT` that guarantees the message the model generates is valid JSON or `JSON_SCHEMA` with a supplied schema that guarantees the model will generate a response that matches your supplied schema.
266266
* xref:api/chat/azure-openai-chat.adoc[Azure OpenAI] - provides a `spring.ai.azure.openai.chat.options.responseFormat` options specifying the format that the model must output. Setting to `{ "type": "json_object" }` enables JSON mode, which guarantees the message the model generates is valid JSON.
267267
* xref:api/chat/ollama-chat.adoc[Ollama] - provides a `spring.ai.ollama.chat.options.format` option to specify the format to return a response in. Currently the only accepted value is `json`.
268268
* xref:api/chat/mistralai-chat.adoc[Mistral AI] - provides a `spring.ai.mistralai.chat.options.responseFormat` option to specify the format to return a response in. Setting to `{ "type": "json_object" }` enables JSON mode, which guarantees the message the model generates is valid JSON.

0 commit comments

Comments
 (0)