Micrometer instrumentation for tool calling #3066
Conversation
|
FYI @shakuzen @jonatan-ivanov Standard Micrometer-based instrumentation following the patterns we used for rest of the project. Some of the shared attributes come from naming conventions originally based on the OTel SemConv. If it's gonna be decided to rename them, then this new instrumentation will use those new names automatically since they are based on enums (i.e. not much refactoring needed). |
|
|
||
| @Bean | ||
| @ConditionalOnMissingBean | ||
| @ConditionalOnProperty(prefix = ToolCallingProperties.CONFIG_PREFIX + ".observations", name = "include-content", |
There was a problem hiding this comment.
fyi: in otel this I believe would be the same ENV var used for normal content. OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true as there is no difference in toggle whether it is a message or a tool. So, for folks using otel they need to map two properties to that.
There was a problem hiding this comment.
Thanks for pointing this out! Yeah, I guess that will be needed.
|
@ThomasVitale curious is the schema capture limited to certain inference platforms or for all of them. I like that you have the ability to capture schema because there are some models that act differently depending on how schema are translated. fyi on the otel genai semconv issue on schema capture open-telemetry/semantic-conventions#1760 Another consumer of this data could be openinference (also otel export) which has a stable schema that includes tool .. schema ;) https://github.com/Arize-ai/openinference/blob/076bb7966d44fccdb2ab94e6f379ef4ae22c39b1/spec/semantic_conventions.md?plain=1#L45 cc @mikeldking @axiomofjoy |
Introduce instrumentation for tool calling using the Micrometer Observation API. By default, metadata about tool calling are exported as metrics and traces. Optionally, the actual tool call input and result can be exported as well by enabling the dedicated feature flag. Signed-off-by: Thomas Vitale <ThomasVitale@users.noreply.github.com>
ThomasVitale
force-pushed
the
tool-calling-instrumentation
branch
from
eefb811 to
9a39339
Compare
|
@codefromthecrypt thanks for reviewing this! The schema captured is part of the ToolDefinition in Spring AI. It's defined consistently across providers, and most of them use JSON Schema. There are exceptions, though. For example, Vertex uses some OpenAPI variant instead of JSON Schema (which SpringAI supports). Some providers might also support a subset of JSON Schema, so having this info part of the telemetry should be useful to troubleshoot those scenarios. I'm aware that some platforms have defined their own naming for such attributes (and at some point, also OTel SemConv might have something). They can all be supported by customising the |
|
@ThomasVitale thanks for the code and the commentary. Looking forward to it! |
|
Rebased and merged at 30eb3ce |
Introduce instrumentation for tool calling using the Micrometer Observation API.
By default, metadata about tool calling are exported as metrics and traces. Optionally, the actual tool call input and result can be exported as well by enabling the dedicated feature flag.
Fixes gh-1924