You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/docs/tutorials/deployment/index.md
+37-4Lines changed: 37 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -120,7 +120,13 @@ print(response.json())
120
120
You should see the response like below:
121
121
122
122
```json
123
-
{'status': 'success', 'data': {'reasoning': 'The capital of France is a well-known fact, commonly taught in geography classes and referenced in various contexts. Paris is recognized globally as the capital city, serving as the political, cultural, and economic center of the country.', 'answer': 'The capital of France is Paris.'}}
123
+
{
124
+
"status": "success",
125
+
"data": {
126
+
"reasoning": "The capital of France is a well-known fact, commonly taught in geography classes and referenced in various contexts. Paris is recognized globally as the capital city, serving as the political, cultural, and economic center of the country.",
127
+
"answer": "The capital of France is Paris."
128
+
}
129
+
}
124
130
```
125
131
126
132
## Deploying with MLflow
@@ -140,7 +146,14 @@ Let's spin up the MLflow tracking server, where we will store our DSPy program.
140
146
```
141
147
142
148
Then we can define the DSPy program and log it to the MLflow server. "log" is an overloaded term in MLflow, basically it means
143
-
we store the program information along with environment requirements in the MLflow server. See the code below:
149
+
we store the program information along with environment requirements in the MLflow server. This is done via the `mlflow.dspy.log_model()`
150
+
function, please see the code below:
151
+
152
+
> [!NOTE]
153
+
> As of MLflow 2.22.0, there is a caveat that you must wrap your DSPy program in a custom DSPy Module class when deploying with MLflow.
154
+
> This is because MLflow requires positional arguments while DSPy pre-built modules disallow positional arguments, e.g., `dspy.Predict`
155
+
> or `dspy.ChainOfThought`. To work around this, create a wrapper class that inherits from `dspy.Module` and implement your program's
156
+
> logic in the `forward()` method, as shown in the example below.
@@ -188,7 +210,18 @@ After the program is deployed, you can test it with the following command:
188
210
You should see the response like below:
189
211
190
212
```json
191
-
{"choices": [{"index": 0, "message": {"role": "assistant", "content": "{\"reasoning\": \"The question asks for the sum of 2 and 2. To find the answer, we simply add the two numbers together: 2 + 2 = 4.\", \"answer\": \"4\"}"}, "finish_reason": "stop"}]}
213
+
{
214
+
"choices": [
215
+
{
216
+
"index": 0,
217
+
"message": {
218
+
"role": "assistant",
219
+
"content": "{\"reasoning\": \"The question asks for the sum of 2 and 2. To find the answer, we simply add the two numbers together: 2 + 2 = 4.\", \"answer\": \"4\"}"
220
+
},
221
+
"finish_reason": "stop"
222
+
}
223
+
]
224
+
}
192
225
```
193
226
194
227
For complete guide on how to deploy a DSPy program with MLflow, and how to customize the deployment, please refer to the
0 commit comments