Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.71.1-stable
What's Changed
- add docs for spend logs by @jugaldb in #10913
- [QA/Guardrails] Allow editing guardrail entity configs by @ishaan-jaff in #10918
- [UI QA Guardrails] - Minor UI Fixes by @ishaan-jaff in #10920
- [CI/CD] Fix - linting cryptography error by @ishaan-jaff in #10921
- LiteLLM Stable release notes by @krrishdholakia in #10919
- docs apply guardrail endpoint by @ishaan-jaff in #10923
- [Docs] Pii masking with LiteLLM by @ishaan-jaff in #10924
- Add provider Featherless AI (access to 4200+ models) by @DarinVerheijke in #10596
- fix(vertex_llm_base.py): remove quota_project_id parameter from crede… by @krrishdholakia in #10915
- Enable structured JSON schema outputs in LM Studio by @estsauver in #10929
- update sambanova models and parameters by @jhpiedrahitao in #10900
- Fix Cohere Rerank Provider by @keyute in #10822
- add skip server startup flag to cli by @mohittalele in #10665
- Allow passed in vertex_ai credentials to be authorized_user type by @pselden in #10899
- fix: handle DB_USER, DB_PASSWORD, DB_HOST problem I faced, since this… by @unrealandychan in #10842
- Add keys and members count in team info by @NANDINI-star in #10950
- Update github.md by @dastaiger in #10946
- docs: add llms.txt and llms_full.txt for LLM-friendly scraping by @colesmcintosh in #10961
- [Fix] Invitation Email does not include the invitation link by @ishaan-jaff in #10958
- Enable key reassignment on UI + Show 'updated at' column for key in all keys table by @krrishdholakia in #10960
- Update existing model's model info on UI + Show model access groups when creating keys/teams on UI by @krrishdholakia in #10965
- Validate migrating keys to teams + Fix mistral image url on async translation by @krrishdholakia in #10966
- add cla to docs by @jugaldb in #10963
- [Fix] List Guardrails - Show config.yaml guardrails on litellm ui by @ishaan-jaff in #10959
- fix: vertex show clear exception on failed refresh by @ishaan-jaff in #10969
- [Feature] Add supports_computer_use to the model list by @slytechnical in #10881
- [Feat] - Add Support for Showing Passthrough endpoint Error Logs on LiteLLM UI by @ishaan-jaff in #10990
- added support to credential delete to support slashes in the curl by @jugaldb in #10987
- Add new gemini preview models + Fix cohere v2 embedding 'embedding_types' param mapping by @krrishdholakia in #10991
- Litellm add new gemini models by @krrishdholakia in #10998
- [Feat] Prometheus - Track
routeon proxy_* metrics by @ishaan-jaff in #10992 - fix: default role for JWT authentication by @ishaan-jaff in #10995
- fix(internal_user_endpoints.py): allow resetting spend/max budget on
/user/update+ Fix updating user budget across multiple instances by @krrishdholakia in #10993 - Improve response_id propagation logic and add tests for valid/empty I… by @daarko10 in #11006
- Support vertex_ai global endpoints for chat by @sorenmat in #10658
- Ollama wildcard support by @matthid in #10982
- Spend rounded to 4 decimal points on Teams page by @NANDINI-star in #11013
- Place organization and team create buttons at the top by @NANDINI-star in #10948
- Feat: add xai/grok-3 pricing by @ishaan-jaff in #11028
- [Feat] Add Image Edits Support to LiteLLM by @ishaan-jaff in #11020
- Updating the available VoyageAI models in the docs by @fzowl in #11003
- Fix UI - Call tool renders correctly when no arguments required by @wagnerjt in #11012
- Verbose error on admin add by @jugaldb in #10978
- Spend rounded to 4 for Organizations and Users page by @NANDINI-star in #11023
- Fix: Handle dict objects in Anthropic streaming response by @jgowdy-godaddy in #11032
- feat: add Databricks Llama 4 Maverick model cost by @bepotp in #11008
- Add LiteLLM Managed Files support for finetuning api
createendpoint by @krrishdholakia in #11039 - Fixes the InvitationLink Prisma find_many query by @jmorenoc-o in #11031
- Support passing
prompt_labelto langfuse + Log consistent modelParams to langfuse by @krrishdholakia in #11018 - Fix CI | Added closing tags for + indentation changes by @NANDINI-star in #11046
- Feat: add MCP to Responses API and bump openai python sdk by @ishaan-jaff in #11029
- Model filter on logs by @NANDINI-star in #11048
- [Feat] Add claude-4 model family by @ishaan-jaff in #11060
- Litellm managed file updates combined by @krrishdholakia in #11040
- [Fix] Reliability Fix - Removing code that was creating threads on errors by @ishaan-jaff in #11066
- [Feat] Add Azure AD certificate-based authentication by @ishaan-jaff in #11069
- Update feature_request.yml by @ishaan-jaff in #11078
- adds tzdata (#10796) by @krrishdholakia in #11052
- Fix proxy_cli.py: avoid overriding DATABASE_URL when it’s already provided by @martin-liu in #11076
- feat(helm): Add loadBalancerClass support for LoadBalancer services by @gunjan-solanki in #11064
- Add Azure Mistral Medium 25.05 by @emerzon in #11063
- fix:Databricks Claude 3.7 Sonnet output token cost: $17.85/M by @bepotp in #11007
- Fix/openrouter stream usage id 8913 by @daarko10 in #11004
- feat: add embeddings to CustomLLM by @tokoko in #10980
- Enable switching between custom auth and litellm api key auth + Fix
/customer/updatefor max budgets by @krrishdholakia in #11070 - Validate if user can call
retrieve+list+deletefile by @krrishdholakia in #11081 - [feature] ConfidentAI logging enabled for proxy and sdk by @spike-spiegel-21 in #10649
- Proper github images by @kreatoo in #10927
- Add devstral-small-2505 model to pricing and context window configuration by @xingyaoww in #11103
- [Ci/CD] use n 4 for mapped tests by @ishaan-jaff in #11109
- Fix/background health check by @baejooc in #10887
- [Feat] Add content policy violation error mapping for image edits by @ishaan-jaff in #11113
- Expose
/listand/infoendpoints for Audit Log events by @krrishdholakia in #11102 - [Feat] Use aiohttp transport by default - 97% lower median latency by @ishaan-jaff in #11097
- Logfire - fix(opentelemetry.py): Fix otel proxy server initialization + Return abbreviated key in key not found error (easier clientside debugging) + Ignore invalid deployments on router load by @krrishdholakia in #11091
- feat(handle_jwt.py): map user to team when added via jwt auth by @krrishdholakia in #11108
- fix(ui_sso.py): maintain backwards compatibility for older user id formats + fix existing user email w/ trailing whitespace check + ensure default_internal_user_settings runs on all user new calls by @krrishdholakia in #11106
- fix(route_llm_request.py): map team model from list in route llm request by @krrishdholakia in #11111
- Remove + Check for unsafe
enterprise/folder imports by @krrishdholakia in #11107 - Fix: Add Claude Sonnet 4 and Opus 4 support for reasoning_effort parameter by @keykbd in #1...
Contributors
dalssoft, sorenmat, and 32 other contributors
Assets 4
v1.71.1-nightly
What's Changed
- Logfire - fix(opentelemetry.py): Fix otel proxy server initialization + Return abbreviated key in key not found error (easier clientside debugging) + Ignore invalid deployments on router load by @krrishdholakia in #11091
- feat(handle_jwt.py): map user to team when added via jwt auth by @krrishdholakia in #11108
- fix(ui_sso.py): maintain backwards compatibility for older user id formats + fix existing user email w/ trailing whitespace check + ensure default_internal_user_settings runs on all user new calls by @krrishdholakia in #11106
- fix(route_llm_request.py): map team model from list in route llm request by @krrishdholakia in #11111
- Remove + Check for unsafe
enterprise/folder imports by @krrishdholakia in #11107 - Fix: Add Claude Sonnet 4 and Opus 4 support for reasoning_effort parameter by @keykbd in #11114
- fix(session): correctly place litellm_session_id at root level instead of metadata by @dalssoft in #11088
- fix(model_management_endpoints): clear cache and reload models after update by @jtong99 in #10853
- [Feat] Add /image/edits on LiteLLM by @ishaan-jaff in #11123
- Correctly delete team model alias when team only model is deleted (#… by @krrishdholakia in #11121
- fix: detect and return status codes in streaming responses by @aholmberg in #10962
- Fix passing standard optional params by @krrishdholakia in #11124
- UI QA fix: team viewer should not see create team by @ishaan-jaff in #11127
- [Chore]: feature flag aiohttp transport - users should opt into using aiohttp transport by @ishaan-jaff in #11132
- v1.71.1-stable - notes by @ishaan-jaff in #11133
- Litellm revert redis changes by @krrishdholakia in #11135
- Litellm fix multi instance checks on teams by @krrishdholakia in #11137
New Contributors
- @keykbd made their first contribution in #11114
- @dalssoft made their first contribution in #11088
- @jtong99 made their first contribution in #10853
Full Changelog: v1.71.0-nightly...v1.71.1-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.1-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 271.62186726419185 | 6.123952252359233 | 0.0 | 1832 | 0 | 215.75241199997208 | 1968.6522410000293 |
| Aggregated | Passed ✅ | 250.0 | 271.62186726419185 | 6.123952252359233 | 0.0 | 1832 | 0 | 215.75241199997208 | 1968.6522410000293 |
Contributors
dalssoft, aholmberg, and 4 other contributors
Assets 4
v1.71.0-nightly
What's Changed
- Litellm managed file updates combined by @krrishdholakia in #11040
- [Fix] Reliability Fix - Removing code that was creating threads on errors by @ishaan-jaff in #11066
- [Feat] Add Azure AD certificate-based authentication by @ishaan-jaff in #11069
- Update feature_request.yml by @ishaan-jaff in #11078
- adds tzdata (#10796) by @krrishdholakia in #11052
- Fix proxy_cli.py: avoid overriding DATABASE_URL when it’s already provided by @martin-liu in #11076
- feat(helm): Add loadBalancerClass support for LoadBalancer services by @gunjan-solanki in #11064
- Add Azure Mistral Medium 25.05 by @emerzon in #11063
- fix:Databricks Claude 3.7 Sonnet output token cost: $17.85/M by @bepotp in #11007
- Fix/openrouter stream usage id 8913 by @daarko10 in #11004
- feat: add embeddings to CustomLLM by @tokoko in #10980
- Enable switching between custom auth and litellm api key auth + Fix
/customer/updatefor max budgets by @krrishdholakia in #11070 - Validate if user can call
retrieve+list+deletefile by @krrishdholakia in #11081 - [feature] ConfidentAI logging enabled for proxy and sdk by @spike-spiegel-21 in #10649
- Proper github images by @kreatoo in #10927
- Add devstral-small-2505 model to pricing and context window configuration by @xingyaoww in #11103
- [Ci/CD] use n 4 for mapped tests by @ishaan-jaff in #11109
- Fix/background health check by @baejooc in #10887
- [Feat] Add content policy violation error mapping for image edits by @ishaan-jaff in #11113
- Expose
/listand/infoendpoints for Audit Log events by @krrishdholakia in #11102 - [Feat] Use aiohttp transport by default - 97% lower median latency by @ishaan-jaff in #11097
New Contributors
- @martin-liu made their first contribution in #11076
- @gunjan-solanki made their first contribution in #11064
- @tokoko made their first contribution in #10980
- @spike-spiegel-21 made their first contribution in #10649
- @kreatoo made their first contribution in #10927
- @baejooc made their first contribution in #10887
Full Changelog: v1.70.4-nightly...v1.71.0-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.71.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 310.0 | 321.9030490306742 | 5.995437687618243 | 4.166377779571852 | 1793 | 1246 | 259.8663090000173 | 771.521746000019 |
| Aggregated | Failed ❌ | 310.0 | 321.9030490306742 | 5.995437687618243 | 4.166377779571852 | 1793 | 1246 | 259.8663090000173 | 771.521746000019 |
Contributors
martin-liu, bepotp, and 10 other contributors
Assets 4
v1.70.2.dev6
Full Changelog: v1.70.2-nightly...v1.70.2.dev6
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.2.dev6
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 470.0 | 546.8056516133963 | 5.687025964404996 | 0.0 | 1702 | 0 | 432.45771799996646 | 2108.0635040000006 |
| Aggregated | Failed ❌ | 470.0 | 546.8056516133963 | 5.687025964404996 | 0.0 | 1702 | 0 | 432.45771799996646 | 2108.0635040000006 |
v1.70.4-nightly
Full Changelog: v1.70.2.dev5...v1.70.4-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.4-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 530.0 | 592.8290290780234 | 5.619046633443039 | 0.0 | 1679 | 0 | 480.5262289999632 | 1595.1236809999614 |
| Aggregated | Failed ❌ | 530.0 | 592.8290290780234 | 5.619046633443039 | 0.0 | 1679 | 0 | 480.5262289999632 | 1595.1236809999614 |
v1.70.2.dev5
What's Changed
- [Feature] Add supports_computer_use to the model list by @slytechnical in #10881
- [Feat] - Add Support for Showing Passthrough endpoint Error Logs on LiteLLM UI by @ishaan-jaff in #10990
- added support to credential delete to support slashes in the curl by @jugaldb in #10987
- Add new gemini preview models + Fix cohere v2 embedding 'embedding_types' param mapping by @krrishdholakia in #10991
- Litellm add new gemini models by @krrishdholakia in #10998
- [Feat] Prometheus - Track
routeon proxy_* metrics by @ishaan-jaff in #10992 - fix: default role for JWT authentication by @ishaan-jaff in #10995
- fix(internal_user_endpoints.py): allow resetting spend/max budget on
/user/update+ Fix updating user budget across multiple instances by @krrishdholakia in #10993 - Improve response_id propagation logic and add tests for valid/empty I… by @daarko10 in #11006
- Support vertex_ai global endpoints for chat by @sorenmat in #10658
- Ollama wildcard support by @matthid in #10982
- Spend rounded to 4 decimal points on Teams page by @NANDINI-star in #11013
- Place organization and team create buttons at the top by @NANDINI-star in #10948
- Feat: add xai/grok-3 pricing by @ishaan-jaff in #11028
- [Feat] Add Image Edits Support to LiteLLM by @ishaan-jaff in #11020
- Updating the available VoyageAI models in the docs by @fzowl in #11003
- Fix UI - Call tool renders correctly when no arguments required by @wagnerjt in #11012
- Verbose error on admin add by @jugaldb in #10978
- Spend rounded to 4 for Organizations and Users page by @NANDINI-star in #11023
- Fix: Handle dict objects in Anthropic streaming response by @jgowdy-godaddy in #11032
- feat: add Databricks Llama 4 Maverick model cost by @bepotp in #11008
- Add LiteLLM Managed Files support for finetuning api
createendpoint by @krrishdholakia in #11039 - Fixes the InvitationLink Prisma find_many query by @jmorenoc-o in #11031
- Support passing
prompt_labelto langfuse + Log consistent modelParams to langfuse by @krrishdholakia in #11018 - Fix CI | Added closing tags for + indentation changes by @NANDINI-star in #11046
- Feat: add MCP to Responses API and bump openai python sdk by @ishaan-jaff in #11029
- Model filter on logs by @NANDINI-star in #11048
- [Feat] Add claude-4 model family by @ishaan-jaff in #11060
New Contributors
- @slytechnical made their first contribution in #10881
- @daarko10 made their first contribution in #11006
- @sorenmat made their first contribution in #10658
- @matthid made their first contribution in #10982
- @jgowdy-godaddy made their first contribution in #11032
- @bepotp made their first contribution in #11008
- @jmorenoc-o made their first contribution in #11031
Full Changelog: v1.70.2-nightly...v1.70.2.dev5
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.2.dev5
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 490.0 | 556.7359308349169 | 5.626256137716844 | 0.0 | 1684 | 0 | 437.79858300001706 | 2137.070654000013 |
| Aggregated | Failed ❌ | 490.0 | 556.7359308349169 | 5.626256137716844 | 0.0 | 1684 | 0 | 437.79858300001706 | 2137.070654000013 |
Contributors
sorenmat, matthid, and 11 other contributors
Assets 4
v1.70.2-nightly
Full Changelog: v1.70.1.dev2...v1.70.2-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.2-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 480.0 | 551.6473700786378 | 5.693462012007638 | 0.0 | 1704 | 0 | 435.9962309999901 | 1522.4978100000044 |
| Aggregated | Failed ❌ | 480.0 | 551.6473700786378 | 5.693462012007638 | 0.0 | 1704 | 0 | 435.9962309999901 | 1522.4978100000044 |
What's Changed
- add docs for spend logs by @jugaldb in #10913
- [QA/Guardrails] Allow editing guardrail entity configs by @ishaan-jaff in #10918
- [UI QA Guardrails] - Minor UI Fixes by @ishaan-jaff in #10920
- [CI/CD] Fix - linting cryptography error by @ishaan-jaff in #10921
- LiteLLM Stable release notes by @krrishdholakia in #10919
- docs apply guardrail endpoint by @ishaan-jaff in #10923
- [Docs] Pii masking with LiteLLM by @ishaan-jaff in #10924
- Add provider Featherless AI (access to 4200+ models) by @DarinVerheijke in #10596
- fix(vertex_llm_base.py): remove quota_project_id parameter from crede… by @krrishdholakia in #10915
- Enable structured JSON schema outputs in LM Studio by @estsauver in #10929
- update sambanova models and parameters by @jhpiedrahitao in #10900
- Fix Cohere Rerank Provider by @keyute in #10822
- add skip server startup flag to cli by @mohittalele in #10665
- Allow passed in vertex_ai credentials to be authorized_user type by @pselden in #10899
- fix: handle DB_USER, DB_PASSWORD, DB_HOST problem I faced, since this… by @unrealandychan in #10842
- Add keys and members count in team info by @NANDINI-star in #10950
- Update github.md by @dastaiger in #10946
- docs: add llms.txt and llms_full.txt for LLM-friendly scraping by @colesmcintosh in #10961
- [Fix] Invitation Email does not include the invitation link by @ishaan-jaff in #10958
- Enable key reassignment on UI + Show 'updated at' column for key in all keys table by @krrishdholakia in #10960
- Update existing model's model info on UI + Show model access groups when creating keys/teams on UI by @krrishdholakia in #10965
- Validate migrating keys to teams + Fix mistral image url on async translation by @krrishdholakia in #10966
- add cla to docs by @jugaldb in #10963
- [Fix] List Guardrails - Show config.yaml guardrails on litellm ui by @ishaan-jaff in #10959
- fix: vertex show clear exception on failed refresh by @ishaan-jaff in #10969
New Contributors
- @DarinVerheijke made their first contribution in #10596
- @estsauver made their first contribution in #10929
- @mohittalele made their first contribution in #10665
- @pselden made their first contribution in #10899
- @unrealandychan made their first contribution in #10842
- @dastaiger made their first contribution in #10946
Full Changelog: v1.70.1-stable...v1.70.2-nightly
Contributors
pselden, estsauver, and 11 other contributors
Assets 4
v1.70.1.dev8
What's Changed
- add docs for spend logs by @jugaldb in #10913
- [QA/Guardrails] Allow editing guardrail entity configs by @ishaan-jaff in #10918
- [UI QA Guardrails] - Minor UI Fixes by @ishaan-jaff in #10920
- [CI/CD] Fix - linting cryptography error by @ishaan-jaff in #10921
- LiteLLM Stable release notes by @krrishdholakia in #10919
- docs apply guardrail endpoint by @ishaan-jaff in #10923
- [Docs] Pii masking with LiteLLM by @ishaan-jaff in #10924
- Add provider Featherless AI (access to 4200+ models) by @DarinVerheijke in #10596
- fix(vertex_llm_base.py): remove quota_project_id parameter from crede… by @krrishdholakia in #10915
- Enable structured JSON schema outputs in LM Studio by @estsauver in #10929
- update sambanova models and parameters by @jhpiedrahitao in #10900
- Fix Cohere Rerank Provider by @keyute in #10822
- add skip server startup flag to cli by @mohittalele in #10665
- Allow passed in vertex_ai credentials to be authorized_user type by @pselden in #10899
- fix: handle DB_USER, DB_PASSWORD, DB_HOST problem I faced, since this… by @unrealandychan in #10842
- Add keys and members count in team info by @NANDINI-star in #10950
- Update github.md by @dastaiger in #10946
- docs: add llms.txt and llms_full.txt for LLM-friendly scraping by @colesmcintosh in #10961
- [Fix] Invitation Email does not include the invitation link by @ishaan-jaff in #10958
New Contributors
- @DarinVerheijke made their first contribution in #10596
- @estsauver made their first contribution in #10929
- @mohittalele made their first contribution in #10665
- @pselden made their first contribution in #10899
- @unrealandychan made their first contribution in #10842
- @dastaiger made their first contribution in #10946
Full Changelog: v1.70.1-stable...v1.70.1.dev8
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.1.dev8
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 490.0 | 562.6894453097084 | 5.610163428556303 | 0.0033413719050365115 | 1679 | 1 | 195.35745899997892 | 1568.1852209999647 |
| Aggregated | Failed ❌ | 490.0 | 562.6894453097084 | 5.610163428556303 | 0.0033413719050365115 | 1679 | 1 | 195.35745899997892 | 1568.1852209999647 |
Contributors
pselden, estsauver, and 11 other contributors
Assets 4
v1.70.1.dev6
Full Changelog: v1.70.1.dev4...v1.70.1.dev6
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.1.dev6
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 520.0 | 614.117503553192 | 5.499725111392901 | 0.003343297940056475 | 1645 | 1 | 470.59824999996636 | 40452.63952500005 |
| Aggregated | Failed ❌ | 520.0 | 614.117503553192 | 5.499725111392901 | 0.003343297940056475 | 1645 | 1 | 470.59824999996636 | 40452.63952500005 |
v1.70.1.dev4
Full Changelog: v1.70.1.dev2...v1.70.1.dev4
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.1.dev4
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 500.0 | 594.1888488711803 | 5.578781212015596 | 0.006685178204931811 | 1669 | 2 | 190.37631700001612 | 40183.55002499999 |
| Aggregated | Failed ❌ | 500.0 | 594.1888488711803 | 5.578781212015596 | 0.006685178204931811 | 1669 | 2 | 190.37631700001612 | 40183.55002499999 |