You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/my-website/docs/proxy/prometheus.md
+8-1Lines changed: 8 additions & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -95,7 +95,14 @@ Use this for for tracking per [user, key, team, etc.](virtual_keys)
95
95
96
96
### Initialize Budget Metrics on Startup
97
97
98
-
If you want to initialize the key/team budget metrics on startup, you can set the `prometheus_initialize_budget_metrics` to `true` in the `config.yaml`
98
+
If you want litellm to emit the budget metrics for all keys, teams irrespective of whether they are getting requests or not, set `prometheus_initialize_budget_metrics` to `true` in the `config.yaml`
99
+
100
+
**How this works:**
101
+
102
+
- If the `prometheus_initialize_budget_metrics` is set to `true`
103
+
- Every 5 minutes litellm runs a cron job to read all keys, teams from the database
104
+
- It then emits the budget metrics for each key, team
105
+
- This is used to populate the budget metrics on the `/metrics` endpoint
Copy file name to clipboardExpand all lines: docs/my-website/release_notes/v1.66.0-stable/index.md
+27-25Lines changed: 27 additions & 25 deletions
Original file line number
Diff line number
Diff line change
@@ -46,7 +46,7 @@ v1.66.0-stable is live now, here are the key highlights of this release
46
46
## Key Highlights
47
47
-**Microsoft SSO Auto-sync**: Auto-sync groups and group members from Azure Entra ID to LiteLLM
48
48
-**Unified File IDs**: Use the same file id across LLM API providers.
49
-
-**Realtime API Cost Tracking**: Track cost of realtime api calls
49
+
-**Realtime API Cost Tracking**: Track cost of realtime API calls
50
50
-**xAI grok-3**: Added support for `xai/grok-3` models
51
51
-**Security Fixes**: Fixed [CVE-2025-0330](https://www.cve.org/CVERecord?id=CVE-2025-0330) and [CVE-2024-6825](https://www.cve.org/CVERecord?id=CVE-2024-6825) vulnerabilities
52
52
@@ -62,10 +62,10 @@ Let's dive in.
62
62
Auto-sync groups and members from Azure Entra ID to LiteLLM
63
63
</p>
64
64
65
-
This release adds support for auto-syncing groups and members on Microsoft Entra ID with LiteLLM. This means that litellm proxy administrators can spend less time managing teams and members and LiteLLM handles the following:
65
+
This release adds support for auto-syncing groups and members on Microsoft Entra ID with LiteLLM. This means that LiteLLM proxy administrators can spend less time managing teams and members and LiteLLM handles the following:
66
66
67
-
- Auto-create Teams that existing on Microsoft Entra ID
68
-
- Sync team members on Microsoft Entra ID with LiteLLM Teams
67
+
- Auto-create teams that exist on Microsoft Entra ID
68
+
- Sync team members on Microsoft Entra ID with LiteLLM teams
69
69
70
70
Get started with this [here](https://docs.litellm.ai/docs/tutorials/msft_sso)
71
71
@@ -76,42 +76,42 @@ Get started with this [here](https://docs.litellm.ai/docs/tutorials/msft_sso)
76
76
## New Models / Updated Models
77
77
78
78
- xAI
79
-
1. Added cost tracking for `xai/grok-3` models [PR](https://github.com/BerriAI/litellm/pull/9920)
80
-
2. Added reasoning_effort support for `xai/grok-3-mini-beta` model family [PR](https://github.com/BerriAI/litellm/pull/9932)
79
+
1. Added reasoning_effort support for `xai/grok-3-mini-beta`[Get Started](https://docs.litellm.ai/docs/providers/xai#reasoning-usage)
80
+
2. Added cost tracking for `xai/grok-3` models [PR](https://github.com/BerriAI/litellm/pull/9920)
81
81
82
82
- Hugging Face
83
-
1.Hugging Face - Added inference providers support [Getting Started](https://docs.litellm.ai/docs/providers/huggingface#serverless-inference-providers)
83
+
1. Added inference providers support [Get Started](https://docs.litellm.ai/docs/providers/huggingface#serverless-inference-providers)
1.Databricks - Removed reasoning_effort from parameters [PR](https://github.com/BerriAI/litellm/pull/9811)
102
+
1. Removed reasoning_effort from parameters [PR](https://github.com/BerriAI/litellm/pull/9811)
103
103
2. Fixed custom endpoint check for Databricks [PR](https://github.com/BerriAI/litellm/pull/9925)
104
104
105
105
- General
106
-
1.Function Calling - Handle pydantic base model in message tool calls, handle tools = [], and support fake streaming on tool calls for meta.llama3-3-70b-instruct-v1:0 [PR](https://github.com/BerriAI/litellm/pull/9774)
107
-
2.LiteLLM Proxy - Allow passing `thinking` param to litellm proxy via client sdk [PR](https://github.com/BerriAI/litellm/pull/9386)
108
-
3.Reasoning - Added litellm.supports_reasoning() util to track if an llm supports reasoning [PR](https://github.com/BerriAI/litellm/pull/9923)
106
+
1.Added litellm.supports_reasoning() util to track if an llm supports reasoning [Get Started](https://docs.litellm.ai/docs/providers/anthropic#reasoning)
107
+
2.Function Calling - Handle pydantic base model in message tool calls, handle tools = [], and support fake streaming on tool calls for meta.llama3-3-70b-instruct-v1:0 [PR](https://github.com/BerriAI/litellm/pull/9774)
108
+
3.LiteLLM Proxy - Allow passing `thinking` param to litellm proxy via client sdk [PR](https://github.com/BerriAI/litellm/pull/9386)
109
109
4. Fixed correctly translating 'thinking' param for litellm [PR](https://github.com/BerriAI/litellm/pull/9904)
110
110
111
111
112
112
## Spend Tracking Improvements
113
113
- OpenAI, Azure
114
-
1. Realtime API Cost tracking with token usage metrics in spend logs [PR](https://github.com/BerriAI/litellm/pull/9795)
114
+
1. Realtime API Cost tracking with token usage metrics in spend logs [Get Started](https://docs.litellm.ai/docs/realtime)
115
115
- Anthropic
116
116
1. Fixed Claude Haiku cache read pricing per token [PR](https://github.com/BerriAI/litellm/pull/9834)
117
117
2. Added cost tracking for Claude responses with base_model [PR](https://github.com/BerriAI/litellm/pull/9897)
@@ -134,32 +134,34 @@ Get started with this [here](https://docs.litellm.ai/docs/tutorials/msft_sso)
1. Added Tag/Policy Management. Create routing rules based on request metadata. This allows you to enforce that requests with `tags="private"` only go to specific models. [Get Started](https://docs.litellm.ai/docs/tutorials/tag_management)
1. Added debug route to allow admins to debug SSO JWT fields [PR](https://github.com/BerriAI/litellm/pull/9835)
150
152
2. Added ability to use MSFT Graph API to assign users to teams [PR](https://github.com/BerriAI/litellm/pull/9865)
151
-
3. Connected LiteLLM to Azure Entra ID Enterprise Application [PR](https://github.com/BerriAI/litellm/pull/9872)
153
+
3. Connected litellm to Azure Entra ID Enterprise Application [PR](https://github.com/BerriAI/litellm/pull/9872)
152
154
4. Added ability for admins to set `default_team_params` for when litellm SSO creates default teams [PR](https://github.com/BerriAI/litellm/pull/9895)
153
155
5. Fixed MSFT SSO to use correct field for user email [PR](https://github.com/BerriAI/litellm/pull/9886)
154
-
6. Added UI support for setting Default Team setting when LiteLLM SSO auto creates teams [PR](https://github.com/BerriAI/litellm/pull/9918)
156
+
6. Added UI support for setting Default Team setting when litellm SSO auto creates teams [PR](https://github.com/BerriAI/litellm/pull/9918)
155
157
5. UI Bug Fixes:
156
158
1. Prevented team, key, org, model numerical values changing on scrolling [PR](https://github.com/BerriAI/litellm/pull/9776)
157
159
2. Instantly reflect key and team updates in UI [PR](https://github.com/BerriAI/litellm/pull/9825)
158
160
159
161
## Logging / Guardrail Improvements
160
162
161
163
1. Prometheus:
162
-
- Emit Key and Team Budget metrics on a cron job schedule [PR](https://github.com/BerriAI/litellm/pull/9528)
164
+
- Emit Key and Team Budget metrics on a cron job schedule [Get Started](https://docs.litellm.ai/docs/proxy/prometheus#initialize-budget-metrics-on-startup)
0 commit comments