Skip to content

Commit ff558ce

Browse files
authored
Document auto route provider selection (#1841)
1 parent 8fd1495 commit ff558ce

File tree

2 files changed

+16
-14
lines changed

2 files changed

+16
-14
lines changed

docs/inference-providers/guides/function-calling.md

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -298,20 +298,24 @@ You can specify which inference provider to use for more control over performanc
298298

299299
<hfoptions id="provider-config">
300300

301-
In the OpenAI client, you can specify the provider you want to use for the request by setting the `base_url` parameter.
301+
In the OpenAI client, you can specify the provider you want to use for the request by appending the provider ID to the model parameter as such:
302302

303303
<hfoption id="openai">
304304

305-
```python
305+
```diff
306306
# The OpenAI client automatically routes through Inference Providers
307307
# You can specify provider preferences in your HF settings
308308
client = OpenAI(
309-
+ base_url="https://router.huggingface.co/v1", # automatically select provider based on hf.co/settings/inference-providers
310-
- base_url="https://router.huggingface.co/together/v1", # manually select Together AI
311-
- base_url="https://router.huggingface.co/nebius/v1", # manually select Nebius
309+
base_url="https://router.huggingface.co/v1",
312310
api_key=os.environ["HF_TOKEN"],
313311
)
314312

313+
client.chat.completions.create(
314+
- model="deepseek-ai/DeepSeek-R1-0528", # automatically select provider based on hf.co/settings/inference-providers
315+
+ model="deepseek-ai/DeepSeek-R1-0528:nebius", # manually select Nebius AI
316+
+ model="deepseek-ai/DeepSeek-R1-0528:hyperbolic", # manually select Hyperbolic
317+
...
318+
)
315319
```
316320

317321
</hfoption>

docs/inference-providers/index.md

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ When you build AI applications, it's tough to manage multiple provider APIs, com
3636

3737
**Zero Vendor Lock-in**: Unlike being tied to a single provider's model catalog, you get access to models from Cerebras, Groq, Together AI, Replicate, and more — all through one consistent interface.
3838

39-
**Production-Ready Performance**: Built for enterprise workloads with automatic failover i.e. ~0 downtime, intelligent routing, and the reliability your applications demand.
39+
**Production-Ready Performance**: Built for enterprise workloads with the reliability your applications demand.
4040

4141
Here's what you can build:
4242

@@ -124,7 +124,7 @@ print(completion.choices[0].message)
124124

125125
If you're already using OpenAI's Python client, then you need a **drop-in OpenAI replacement**. Just swap-out the base URL to instantly access hundreds of additional open-weights models through our provider network.
126126

127-
Our system automatically routes your request to the most popular provider for the specified model. You can also order the available providers from your [user settings](https://huggingface.co/settings/inference-providers), in that case we will route your request to the first provider supporting the model.
127+
Our system automatically routes your request to the most popular provider for the specified model. You can also select the provider of your choice by appending it to the model id (e.g. `"deepseek-ai/DeepSeek-V3-0324:sambanova"`).
128128

129129
```python
130130
import os
@@ -152,8 +152,7 @@ completion = client.chat.completions.create(
152152

153153
For maximum control and interoperability with custom frameworks, use our OpenAI-compatible REST API directly.
154154

155-
Our routing system automatically selects the most popular available provider for your chosen model,
156-
or to your preferred provider if you've sorted the providers from your [user settings](https://huggingface.co/settings/inference-providers).
155+
Our routing system automatically selects the most popular available provider for your chosen model. You can also select the provider of your choice by appending it to the model id (e.g. `"deepseek-ai/DeepSeek-V3-0324:novita"`).
157156

158157
```python
159158
import os
@@ -219,7 +218,7 @@ console.log(chatCompletion.choices[0].message);
219218

220219
<hfoption id="openai">
221220

222-
If you're already using OpenAI's Javascript client, then you need a **drop-in OpenAI replacement**. Just swap-out the base URL to instantly access hundreds of additional open-weights models through our provider network.
221+
If you're already using OpenAI's Javascript client, then you need a **drop-in OpenAI replacement**. Just swap-out the base URL to instantly access hundreds of additional open-weights models through our provider network. Our system automatically routes your request to the most popular provider for the specified model. You can also select the provider of your choice by appending it to the model id (e.g. `"deepseek-ai/DeepSeek-V3-0324:nebius"`).
223222

224223
```javascript
225224
import OpenAI from "openai";
@@ -248,8 +247,7 @@ console.log(completion.choices[0].message.content);
248247

249248
For lightweight applications or custom implementations, use our REST API directly with standard fetch.
250249

251-
Our routing system automatically selects the most popular available provider for your chosen model,
252-
or to your preferred provider if you've sorted the providers from your [user settings](https://huggingface.co/settings/inference-providers).
250+
Our routing system automatically selects the most popular available provider for your chosen model. You can also select the provider of your choice by appending it to the model id (e.g. `"deepseek-ai/DeepSeek-V3-0324:fireworks-ai"`).
253251

254252
```js
255253
import fetch from "node-fetch";
@@ -282,8 +280,8 @@ console.log(await response.json());
282280

283281
#### HTTP / cURL
284282

285-
For testing, debugging, or integrating with any HTTP client, here's the raw REST API format. Our intelligent routing automatically selects the most popular provider for your requested model,
286-
or to your preferred provider if you've sorted the providers from your [user settings](https://huggingface.co/settings/inference-providers).
283+
For testing, debugging, or integrating with any HTTP client, here's the raw REST API format.
284+
Our routing system automatically selects the most popular available provider for your chosen model. You can also select the provider of your choice by appending it to the model id (e.g. `"deepseek-ai/DeepSeek-V3-0324:fireworks-ai"`).
287285

288286
```bash
289287
curl https://router.huggingface.co/v1/chat/completions \

0 commit comments

Comments
 (0)