Skip to content

Commit 65e18f6

Browse files
docs(index.md): update changelog with realtime api cost tracking details
1 parent 4436838 commit 65e18f6

File tree

4 files changed

+37
-24
lines changed

4 files changed

+37
-24
lines changed

docs/my-website/docs/proxy/cost_tracking.md

Lines changed: 0 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -205,28 +205,6 @@ curl -X POST \
205205
{"message":"Spend for all API Keys and Teams reset successfully","status":"success"}
206206
```
207207

208-
209-
## Set 'base_model' for Cost Tracking (e.g. Azure deployments)
210-
211-
**Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking
212-
213-
**Solution** ✅ : Set `base_model` on your config so litellm uses the correct model for calculating azure cost
214-
215-
Get the base model name from [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
216-
217-
Example config with `base_model`
218-
```yaml
219-
model_list:
220-
- model_name: azure-gpt-3.5
221-
litellm_params:
222-
model: azure/chatgpt-v-2
223-
api_base: os.environ/AZURE_API_BASE
224-
api_key: os.environ/AZURE_API_KEY
225-
api_version: "2023-07-01-preview"
226-
model_info:
227-
base_model: azure/gpt-4-1106-preview
228-
```
229-
230208
## Daily Spend Breakdown API
231209

232210
Retrieve granular daily usage data for a user (by model, provider, and API key) with a single endpoint.

docs/my-website/docs/proxy/custom_pricing.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,28 @@ model_list:
8383
cache_read_input_token_cost: 0.0000006
8484
```
8585

86+
## Set 'base_model' for Cost Tracking (e.g. Azure deployments)
87+
88+
**Problem**: Azure returns `gpt-4` in the response when `azure/gpt-4-1106-preview` is used. This leads to inaccurate cost tracking
89+
90+
**Solution** ✅ : Set `base_model` on your config so litellm uses the correct model for calculating azure cost
91+
92+
Get the base model name from [here](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
93+
94+
Example config with `base_model`
95+
```yaml
96+
model_list:
97+
- model_name: azure-gpt-3.5
98+
litellm_params:
99+
model: azure/chatgpt-v-2
100+
api_base: os.environ/AZURE_API_BASE
101+
api_key: os.environ/AZURE_API_KEY
102+
api_version: "2023-07-01-preview"
103+
model_info:
104+
base_model: azure/gpt-4-1106-preview
105+
```
106+
107+
86108
## Debugging
87109

88110
If you're custom pricing is not being used or you're seeing errors, please check the following:

docs/my-website/img/realtime_api.png

182 KB
Loading

docs/my-website/release_notes/v1.66.0-stable/index.md

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ authors:
1212
url: https://www.linkedin.com/in/reffajnaahsi/
1313
image_url: https://pbs.twimg.com/profile_images/1613813310264340481/lz54oEiB_400x400.jpg
1414

15-
tags: []
15+
tags: ["sso", "unified_file_id", "cost_tracking", "security"]
1616
hide_table_of_contents: false
1717
---
1818

@@ -69,7 +69,20 @@ This release adds support for auto-syncing groups and members on Microsoft Entra
6969

7070
Get started with this [here](https://docs.litellm.ai/docs/tutorials/msft_sso)
7171

72-
## Unified File ID
72+
## Realtime API Cost Tracking
73+
74+
<Image
75+
img={require('../../img/realtime_api.png')}
76+
style={{width: '100%', display: 'block'}}
77+
/>
78+
79+
80+
This release adds Realtime API logging + cost tracking.
81+
- **Logging**: LiteLLM now logs the complete response from realtime calls to all logging integrations (DB, S3, Langfuse, etc.)
82+
- **Cost Tracking**: You can now set 'base_model' and custom pricing for realtime models. [Custom Pricing](../../docs/proxy/custom_pricing)
83+
- **Budgets**: Your key/user/team budgets now work for realtime models as well.
84+
85+
Start [here](https://docs.litellm.ai/docs/realtime)
7386

7487

7588

0 commit comments

Comments
 (0)