Skip to content

Commit d211306

Browse files
Add direct public API (#1599)
Co-authored-by: Alex Hall <alex.mojaki@gmail.com>
1 parent 89cf4d9 commit d211306

File tree

11 files changed

+528
-14
lines changed

11 files changed

+528
-14
lines changed

docs/api/direct.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# `pydantic_ai.direct`
2+
3+
::: pydantic_ai.direct

docs/direct.md

Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
# Direct Model Requests
2+
3+
The `direct` module provides low-level methods for making imperative requests to LLMs where the only abstraction is input and output schema translation, enabling you to use all models with the same API.
4+
5+
These methods are thin wrappers around the [`Model`][pydantic_ai.models.Model] implementations, offering a simpler interface when you don't need the full functionality of an [`Agent`][pydantic_ai.Agent].
6+
7+
The following functions are available:
8+
9+
- [`model_request`][pydantic_ai.direct.model_request]: Make a non-streamed async request to a model
10+
- [`model_request_sync`][pydantic_ai.direct.model_request_sync]: Make a non-streamed synchronous request to a model
11+
- [`model_request_stream`][pydantic_ai.direct.model_request_stream]: Make a streamed async request to a model
12+
13+
## Basic Example
14+
15+
Here's a simple example demonstrating how to use the direct API to make a basic request:
16+
17+
```python title="direct_basic.py"
18+
from pydantic_ai.direct import model_request_sync
19+
from pydantic_ai.messages import ModelRequest
20+
21+
# Make a synchronous request to the model
22+
model_response = model_request_sync(
23+
'anthropic:claude-3-5-haiku-latest',
24+
[ModelRequest.user_text_prompt('What is the capital of France?')]
25+
)
26+
27+
print(model_response.parts[0].content)
28+
#> Paris
29+
print(model_response.usage)
30+
"""
31+
Usage(requests=1, request_tokens=56, response_tokens=1, total_tokens=57, details=None)
32+
"""
33+
```
34+
35+
_(This example is complete, it can be run "as is")_
36+
37+
## Advanced Example with Tool Calling
38+
39+
You can also use the direct API to work with function/tool calling.
40+
41+
Even here we can use Pydantic to generate the JSON schema for the tool:
42+
43+
```python
44+
from pydantic import BaseModel
45+
from typing_extensions import Literal
46+
47+
from pydantic_ai.direct import model_request
48+
from pydantic_ai.messages import ModelRequest
49+
from pydantic_ai.models import ModelRequestParameters
50+
from pydantic_ai.tools import ToolDefinition
51+
52+
53+
class Divide(BaseModel):
54+
"""Divide two numbers."""
55+
56+
numerator: float
57+
denominator: float
58+
on_inf: Literal['error', 'infinity'] = 'infinity'
59+
60+
61+
async def main():
62+
# Make a request to the model with tool access
63+
model_response = await model_request(
64+
'openai:gpt-4.1-nano',
65+
[ModelRequest.user_text_prompt('What is 123 / 456?')],
66+
model_request_parameters=ModelRequestParameters(
67+
function_tools=[
68+
ToolDefinition(
69+
name=Divide.__name__.lower(),
70+
description=Divide.__doc__ or '',
71+
parameters_json_schema=Divide.model_json_schema(),
72+
)
73+
],
74+
allow_text_output=True, # Allow model to either use tools or respond directly
75+
),
76+
)
77+
print(model_response)
78+
"""
79+
ModelResponse(
80+
parts=[
81+
ToolCallPart(
82+
tool_name='divide',
83+
args={'numerator': '123', 'denominator': '456'},
84+
tool_call_id='pyd_ai_2e0e396768a14fe482df90a29a78dc7b',
85+
part_kind='tool-call',
86+
)
87+
],
88+
usage=Usage(
89+
requests=1,
90+
request_tokens=55,
91+
response_tokens=7,
92+
total_tokens=62,
93+
details=None,
94+
),
95+
model_name='gpt-4.1-nano',
96+
timestamp=datetime.datetime(...),
97+
kind='response',
98+
)
99+
"""
100+
```
101+
102+
_(This example is complete, it can be run "as is" — you'll need to add `asyncio.run(main())` to run `main`)_
103+
104+
## When to Use the direct API vs Agent
105+
106+
The direct API is ideal when:
107+
108+
1. You need more direct control over model interactions
109+
2. You want to implement custom behavior around model requests
110+
3. You're building your own abstractions on top of model interactions
111+
112+
For most application use cases, the higher-level [`Agent`][pydantic_ai.Agent] API provides a more convenient interface with additional features such as built-in tool execution, retrying, structured output parsing, and more.
113+
114+
## OpenTelemetry or Logfire Instrumentation
115+
116+
As with [agents][pydantic_ai.Agent], you can enable OpenTelemetry/Logfire instrumentation with just a few extra lines
117+
118+
```python {title="direct_instrumented.py" hl_lines="1 6 7"}
119+
import logfire
120+
121+
from pydantic_ai.direct import model_request_sync
122+
from pydantic_ai.messages import ModelRequest
123+
124+
logfire.configure()
125+
logfire.instrument_pydantic_ai()
126+
127+
# Make a synchronous request to the model
128+
model_response = model_request_sync(
129+
'anthropic:claude-3-5-haiku-latest',
130+
[ModelRequest.user_text_prompt('What is the capital of France?')],
131+
)
132+
133+
print(model_response.parts[0].content)
134+
#> Paris
135+
```
136+
137+
_(This example is complete, it can be run "as is")_
138+
139+
You can also enable OpenTelemetry on a per call basis:
140+
141+
```python {title="direct_instrumented.py" hl_lines="1 6 12"}
142+
import logfire
143+
144+
from pydantic_ai.direct import model_request_sync
145+
from pydantic_ai.messages import ModelRequest
146+
147+
logfire.configure()
148+
149+
# Make a synchronous request to the model
150+
model_response = model_request_sync(
151+
'anthropic:claude-3-5-haiku-latest',
152+
[ModelRequest.user_text_prompt('What is the capital of France?')],
153+
instrument=True
154+
)
155+
156+
print(model_response.parts[0].content)
157+
#> Paris
158+
```
159+
160+
See [Debugging and Monitoring](logfire.md) for more details, including how to instrument with plain OpenTelemetry without Logfire.

mkdocs.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ nav:
3838
- graph.md
3939
- evals.md
4040
- input.md
41+
- direct.md
4142
- MCP:
4243
- mcp/index.md
4344
- mcp/client.md
@@ -68,6 +69,7 @@ nav:
6869
- api/usage.md
6970
- api/mcp.md
7071
- api/format_as_xml.md
72+
- api/direct.md
7173
- api/models/base.md
7274
- api/models/openai.md
7375
- api/models/anthropic.md

pydantic_ai_slim/pydantic_ai/agent.py

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@
2828
result,
2929
usage as _usage,
3030
)
31-
from .models.instrumented import InstrumentationSettings, InstrumentedModel
31+
from .models.instrumented import InstrumentationSettings, InstrumentedModel, instrument_model
3232
from .result import FinalResult, OutputDataT, StreamedRunResult, ToolOutput
3333
from .settings import ModelSettings, merge_model_settings
3434
from .tools import (
@@ -108,7 +108,7 @@ class Agent(Generic[AgentDepsT, OutputDataT]):
108108
model: models.Model | models.KnownModelName | str | None
109109
"""The default model configured for this agent.
110110
111-
We allow str here since the actual list of allowed models changes frequently.
111+
We allow `str` here since the actual list of allowed models changes frequently.
112112
"""
113113

114114
name: str | None
@@ -233,7 +233,7 @@ def __init__(
233233
234234
Args:
235235
model: The default model to use for this agent, if not provide,
236-
you must provide the model when calling it. We allow str here since the actual list of allowed models changes frequently.
236+
you must provide the model when calling it. We allow `str` here since the actual list of allowed models changes frequently.
237237
output_type: The type of the output data, used to validate the data returned by the model,
238238
defaults to `str`.
239239
instructions: Instructions to use for this agent, you can also register instructions via a function with
@@ -1582,13 +1582,7 @@ def _get_model(self, model: models.Model | models.KnownModelName | str | None) -
15821582
if instrument is None:
15831583
instrument = self._instrument_default
15841584

1585-
if instrument and not isinstance(model_, InstrumentedModel):
1586-
if instrument is True:
1587-
instrument = InstrumentationSettings()
1588-
1589-
model_ = InstrumentedModel(model_, instrument)
1590-
1591-
return model_
1585+
return instrument_model(model_, instrument)
15921586

15931587
def _get_deps(self: Agent[T, OutputDataT], deps: T) -> T:
15941588
"""Get deps for a run.

0 commit comments

Comments
 (0)