You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs: Clarify function calling concepts and usage patterns
Improves function calling documentation with clearer organization and examples:
- Reorganizes content into server-side and client-side registration sections
- Adds detailed examples for both function-invoking and method-invoking approaches
- Enhances tool context documentation with diagrams and usage examples
- Updates function-calling diagrams to reflect current implementation
Copy file name to clipboardExpand all lines: spring-ai-core/src/test/java/org/springframework/ai/model/function/MethodInvokingFunctionCallbackTests.java
Copy file name to clipboardExpand all lines: spring-ai-docs/src/main/antora/modules/ROOT/pages/api/functions.adoc
+69-24Lines changed: 69 additions & 24 deletions
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ In general, the custom functions need to provide a function `name`, `descriptio
28
28
29
29
As a developer, you need to implement a function that takes the function call arguments sent from the AI model, and responds with the result back to the model. Your function can in turn invoke other 3rd party services to provide the results.
30
30
31
-
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatClient`.
31
+
Spring AI makes this as easy as defining a `@Bean` definition that returns a `java.util.Function` and supplying the bean name as an option when invoking the `ChatClient` or registering the function dynamically in your prompt request.
32
32
33
33
Under the hood, Spring wraps your POJO (the function) with the appropriate adapter code that enables interaction with the AI Model, saving you from writing tedious boilerplate code.
34
34
The basis of the underlying infrastructure is the link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/model/function/FunctionCallback.java[FunctionCallback.java] interface and the companion Builder utility class to simplify the implementation and registration of Java callback functions.
@@ -39,16 +39,20 @@ Suppose we want the AI model to respond with information that it does not have,
39
39
40
40
We can provide the AI model with metadata about our own functions that it can use to retrieve that information as it processes your prompt.
41
41
42
-
For example, if during the processing of a prompt, the AI Model determines that it needs additional information about the temperature in a given location, it will start a server-side generated request/response interaction. The AI Model invokes a client side function.
43
-
The AI Model provides method invocation details as JSON, and it is the responsibility of the client to execute that function and return the response.
42
+
For example, if during the processing of a prompt, the AI Model determines that it needs additional information about the temperature in a given location, it will start a server-side generated request/response interaction.
43
+
Instead of returning the final response message, the AI Model returns at special Tool Call request, providing the function name and arguments (as JSON).
44
+
It is the responsibility of the client to process this message and execute the named function and return the response
45
+
as Tool Response message back to the AI Model.
44
46
45
47
Spring AI greatly simplifies the code you need to write to support function invocation.
46
48
It brokers the function invocation conversation for you.
47
-
You can simply provide your function definition as a `@Bean` and then provide the bean name of the function in your prompt options.
49
+
You can simply provide your function definition as a `@Bean` and then provide the bean name of the function in your prompt options or pass the function directly as a parameter in your prompt request options.
50
+
48
51
You can also reference multiple function bean names in your prompt.
49
52
50
-
== Quick Start
53
+
=== Example Use Case
51
54
55
+
Lets define a simple use case that we can use as an example to explain how function invocation works.
52
56
Let's create a chatbot that answer questions by calling our own function.
53
57
To support the response of the chatbot, we will register our own function that takes a location and returns the current weather in that location.
54
58
@@ -91,7 +95,9 @@ data class Response(val temp: Double, val unit: Unit) {}
91
95
======
92
96
--
93
97
94
-
=== Registering Functions as Beans
98
+
== Server-Side Registration
99
+
100
+
=== Functions as Beans
95
101
96
102
Spring AI provides multiple ways to register custom functions as beans in the Spring context.
97
103
@@ -266,18 +272,23 @@ Here is the current weather for the requested cities:
266
272
267
273
The link:https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/openai/tool/FunctionCallbackWithPlainFunctionBeanIT.java[FunctionCallbackWithPlainFunctionBeanIT.java] test demo this approach.
268
274
269
-
=== Register functions: On the fly
275
+
== Client-Side Registration
276
+
277
+
In addition to the auto-configuration, you can register callback functions, dynamically.
278
+
You can use either the function invoking or method invoking approaches to register functions with your `ChatClient` or `ChatModel` requests.
270
279
271
-
In addition to the auto-configuration, you can register callback functions, dynamically:
280
+
The client-side registration enables you to register functions by default.
281
+
282
+
=== Function Invoking
272
283
273
284
[source,java]
274
285
----
275
286
ChatClient chatClient = ...
276
-
287
+
277
288
ChatResponse response = this.chatClient.prompt("What's the weather like in San Francisco, Tokyo, and Paris?")
278
289
.functions(FunctionCallback.builder()
279
290
.description("Get the weather in location") // (2) function description
280
-
.function("CurrentWeather", new MockWeatherService()) // (1) function name and instance
291
+
.function("currentWeather", (Request request) -> new Response(30.0, Unit.C)) // (1) function name and instance
281
292
.inputType(MockWeatherService.Request.class) // (3) input type to build the JSON schema
282
293
.build())
283
294
.call()
@@ -290,29 +301,28 @@ This approach allows to choose dynamically different functions to be called base
290
301
291
302
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-spring-boot-autoconfigure/src/test/java/org/springframework/ai/autoconfigure/openai/tool/FunctionCallbackInPromptIT.java[FunctionCallbackInPromptIT.java] integration test provides a complete example of how to register a function with the `ChatClient` and use it in a prompt request.
The `MethodFunctionCallback` enables method invocation through reflection while automatically handling JSON schema generation and parameter conversion.
306
+
The `MethodInvokingFunctionCallback` enables method invocation through reflection while automatically handling JSON schema generation and parameter conversion.
296
307
It's particularly useful for integrating Java methods as callable functions within AI model interactions.
297
308
298
-
The `MethodFunctionCallback` implements the `FunctionCallback` interface and provides:
309
+
The `MethodInvokingFunctionCallback` implements the `FunctionCallback` interface and provides:
299
310
300
311
- Automatic JSON schema generation for method parameters
301
312
- Support for both static and instance methods
302
313
- Any number of parameters (including none) and return values (including void)
303
314
- Any parameter/return types (primitives, objects, collections)
304
315
- Special handling for `ToolContext` parameters
305
316
306
-
You need the `FunctionCallback.Builder` to create `MethodFunctionCallback` like this:
317
+
You need the `FunctionCallback.Builder` to create `MethodInvokingFunctionCallback` like this:
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/chat/client/OpenAiChatClientMethodFunctionCallbackIT.java[OpenAiChatClientMethodFunctionCallbackIT]
378
+
The https://github.com/spring-projects/spring-ai/blob/main/models/spring-ai-openai/src/test/java/org/springframework/ai/openai/chat/client/OpenAiChatClientMethodInvokingFunctionCallbackIT.java[OpenAiChatClientMethodInvokingFunctionCallbackIT]
369
379
integration test provides additional examples of how to use the FunctionCallback.Builder to create method invocation FunctionCallbacks.
370
380
371
-
=== Tool Context
381
+
== Tool Context
382
+
383
+
Spring AI now supports passing additional contextual information to function callbacks through a tool context.
384
+
This feature allows you to provide extra, user provided, data that can be used within the function execution along with the function arguments passed by the AI model.
385
+
386
+
image::function-calling-tool-context.jpg[Function calling with Tool Context, width=700, align="center"]
372
387
373
-
Spring AI now supports passing additional contextual information to function callbacks through a tool context. This feature allows you to provide extra data that can be used within the function execution, enhancing the flexibility and power of function calling.
388
+
The https://github.com/spring-projects/spring-ai/blob/main/spring-ai-core/src/main/java/org/springframework/ai/chat/model/ToolContext.java[ToolContext] class provides a way to pass additional context information.
374
389
375
-
The context information that is passed in as the second argument of a `java.util.BiFunction`. The `ToolContext` contains as an immutable `Map<String,Object>` allowing you to access key-value pairs.
390
+
=== Using Tool Context
376
391
377
-
==== How to Use Tool Context
392
+
In case of function-invoking, the context information that is passed in as the second argument of a `java.util.BiFunction`.
393
+
394
+
For method-invoking, the context information is passed as a method argument of type `ToolContext`.
395
+
396
+
==== Function Invoking
378
397
379
398
You can set the tool context when building your chat options and use a BiFunction for your callback:
380
399
@@ -415,3 +434,29 @@ ChatResponse response = chatClient.prompt("What's the weather like in San Franci
415
434
In this example, the `weatherFunction` is defined as a BiFunction that takes both the request and the tool context as parameters. This allows you to access the context directly within the function logic.
416
435
417
436
This approach allows you to pass session-specific or user-specific information to your functions, enabling more contextual and personalized responses.
437
+
438
+
==== Method Invoking
439
+
440
+
[source,java]
441
+
----
442
+
public class DeviceController {
443
+
public void setDeviceState(String deviceId, boolean state, ToolContext context) {
0 commit comments