Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.70.1.dev8
What's Changed
- add docs for spend logs by @jugaldb in #10913
- [QA/Guardrails] Allow editing guardrail entity configs by @ishaan-jaff in #10918
- [UI QA Guardrails] - Minor UI Fixes by @ishaan-jaff in #10920
- [CI/CD] Fix - linting cryptography error by @ishaan-jaff in #10921
- LiteLLM Stable release notes by @krrishdholakia in #10919
- docs apply guardrail endpoint by @ishaan-jaff in #10923
- [Docs] Pii masking with LiteLLM by @ishaan-jaff in #10924
- Add provider Featherless AI (access to 4200+ models) by @DarinVerheijke in #10596
- fix(vertex_llm_base.py): remove quota_project_id parameter from crede… by @krrishdholakia in #10915
- Enable structured JSON schema outputs in LM Studio by @estsauver in #10929
- update sambanova models and parameters by @jhpiedrahitao in #10900
- Fix Cohere Rerank Provider by @keyute in #10822
- add skip server startup flag to cli by @mohittalele in #10665
- Allow passed in vertex_ai credentials to be authorized_user type by @pselden in #10899
- fix: handle DB_USER, DB_PASSWORD, DB_HOST problem I faced, since this… by @unrealandychan in #10842
- Add keys and members count in team info by @NANDINI-star in #10950
- Update github.md by @dastaiger in #10946
- docs: add llms.txt and llms_full.txt for LLM-friendly scraping by @colesmcintosh in #10961
- [Fix] Invitation Email does not include the invitation link by @ishaan-jaff in #10958
New Contributors
- @DarinVerheijke made their first contribution in #10596
- @estsauver made their first contribution in #10929
- @mohittalele made their first contribution in #10665
- @pselden made their first contribution in #10899
- @unrealandychan made their first contribution in #10842
- @dastaiger made their first contribution in #10946
Full Changelog: v1.70.1-stable...v1.70.1.dev8
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.1.dev8
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 490.0 | 562.6894453097084 | 5.610163428556303 | 0.0033413719050365115 | 1679 | 1 | 195.35745899997892 | 1568.1852209999647 |
| Aggregated | Failed ❌ | 490.0 | 562.6894453097084 | 5.610163428556303 | 0.0033413719050365115 | 1679 | 1 | 195.35745899997892 | 1568.1852209999647 |
Contributors
pselden, estsauver, and 11 other contributors
Assets 4
v1.70.1.dev6
Full Changelog: v1.70.1.dev4...v1.70.1.dev6
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.1.dev6
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 520.0 | 614.117503553192 | 5.499725111392901 | 0.003343297940056475 | 1645 | 1 | 470.59824999996636 | 40452.63952500005 |
| Aggregated | Failed ❌ | 520.0 | 614.117503553192 | 5.499725111392901 | 0.003343297940056475 | 1645 | 1 | 470.59824999996636 | 40452.63952500005 |
v1.70.1.dev4
Full Changelog: v1.70.1.dev2...v1.70.1.dev4
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.1.dev4
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 500.0 | 594.1888488711803 | 5.578781212015596 | 0.006685178204931811 | 1669 | 2 | 190.37631700001612 | 40183.55002499999 |
| Aggregated | Failed ❌ | 500.0 | 594.1888488711803 | 5.578781212015596 | 0.006685178204931811 | 1669 | 2 | 190.37631700001612 | 40183.55002499999 |
v1.70.1.dev2
Full Changelog: v1.67.0-stable.patch2...v1.70.1.dev2
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.1.dev2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 530.0 | 603.0679202423511 | 5.573003547206886 | 0.0 | 1667 | 0 | 488.8812669999538 | 2023.6071630000083 |
| Aggregated | Failed ❌ | 530.0 | 603.0679202423511 | 5.573003547206886 | 0.0 | 1667 | 0 | 488.8812669999538 | 2023.6071630000083 |
v1.70.1.dev11
Full Changelog: v1.70.1.dev8...v1.70.1.dev11
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.1.dev11
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 510.0 | 588.5182217067622 | 5.584563874858318 | 0.0 | 1671 | 0 | 469.68324099998426 | 1859.8325959999897 |
| Aggregated | Failed ❌ | 510.0 | 588.5182217067622 | 5.584563874858318 | 0.0 | 1671 | 0 | 469.68324099998426 | 1859.8325959999897 |
v1.67.0-stable.patch2
What's Changed
- add docs for spend logs by @jugaldb in #10913
- [QA/Guardrails] Allow editing guardrail entity configs by @ishaan-jaff in #10918
- [UI QA Guardrails] - Minor UI Fixes by @ishaan-jaff in #10920
- [CI/CD] Fix - linting cryptography error by @ishaan-jaff in #10921
- LiteLLM Stable release notes by @krrishdholakia in #10919
- docs apply guardrail endpoint by @ishaan-jaff in #10923
- [Docs] Pii masking with LiteLLM by @ishaan-jaff in #10924
- Add provider Featherless AI (access to 4200+ models) by @DarinVerheijke in #10596
- fix(vertex_llm_base.py): remove quota_project_id parameter from crede… by @krrishdholakia in #10915
- Enable structured JSON schema outputs in LM Studio by @estsauver in #10929
- update sambanova models and parameters by @jhpiedrahitao in #10900
- Fix Cohere Rerank Provider by @keyute in #10822
- add skip server startup flag to cli by @mohittalele in #10665
- Allow passed in vertex_ai credentials to be authorized_user type by @pselden in #10899
- fix: handle DB_USER, DB_PASSWORD, DB_HOST problem I faced, since this… by @unrealandychan in #10842
- Add keys and members count in team info by @NANDINI-star in #10950
- Update github.md by @dastaiger in #10946
- docs: add llms.txt and llms_full.txt for LLM-friendly scraping by @colesmcintosh in #10961
- [Fix] Invitation Email does not include the invitation link by @ishaan-jaff in #10958
- Enable key reassignment on UI + Show 'updated at' column for key in all keys table by @krrishdholakia in #10960
- Update existing model's model info on UI + Show model access groups when creating keys/teams on UI by @krrishdholakia in #10965
- Validate migrating keys to teams + Fix mistral image url on async translation by @krrishdholakia in #10966
- add cla to docs by @jugaldb in #10963
- [Fix] List Guardrails - Show config.yaml guardrails on litellm ui by @ishaan-jaff in #10959
- fix: vertex show clear exception on failed refresh by @ishaan-jaff in #10969
New Contributors
- @DarinVerheijke made their first contribution in #10596
- @estsauver made their first contribution in #10929
- @mohittalele made their first contribution in #10665
- @pselden made their first contribution in #10899
- @unrealandychan made their first contribution in #10842
- @dastaiger made their first contribution in #10946
Full Changelog: v1.70.1-stable...v1.67.0-stable.patch2
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.67.0-stable.patch2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 261.5294727693982 | 6.120234747528359 | 0.0 | 1830 | 0 | 217.36453699998037 | 1204.8032490000082 |
| Aggregated | Passed ✅ | 250.0 | 261.5294727693982 | 6.120234747528359 | 0.0 | 1830 | 0 | 217.36453699998037 | 1204.8032490000082 |
Contributors
pselden, estsauver, and 11 other contributors
Assets 4
v1.70.1-stable
What's Changed
- Handle gemini audio input by @krrishdholakia in #10739
- Fixed Ollama Structured Response not working #10616 by @imdigitalashish in #10617
- fix(factory.py): Add reasoning content handling for missing assistant… by @LouisShark in #10688
- [Feat] Add tools support for Nvidia NIM by @ishaan-jaff in #10763
- [Fix]: /messages - allow using dynamic AWS params by @ishaan-jaff in #10769
- fix: pass application/json for GenericAPILogger by @ishaan-jaff in #10772
- [Docs] Using litellm with Google ADK by @ishaan-jaff in #10777
- Update Nscale model providers to point to website by @OscarSavNS in #10764
- [Fix] Allow using dynamic aws_region with /messages on Bedrock by @ishaan-jaff in #10779
- [Feat] Option to force/always use the litellm proxy (#10559) (#10633) by @ishaan-jaff in #10773
- feat: Addded EU Anthropic Inference profile for Claude 3.7 by @wagnerjt in #10767
- Add new model provider Novita AI (#7582) by @krrishdholakia in #9527
- Support Anthropic web search tool + Add more google finish reason mapping by @krrishdholakia in #10785
- Fix azure dall e 3 call with custom model name + Handle
Bearer $LITELLM_API_KEYinx-litellm-api-keycustom header by @krrishdholakia in #10776 - [Refactor] Move LLM Guard, Secret Detection to Enterprise Pip packagea by @ishaan-jaff in #10782
- fix: URL encode OTEL_EXPORTER_OTLP_TRACES_HEADERS for Phoenix Integration by @arizedatngo in #10654
- [README] Add Llama API in supported providers table by @seyeong-han in #10802
- [Fix] Maximum Vertex Recursion Depth Reached when transforming response_format by @ishaan-jaff in #10798
- add ollama api base defaults by @jugaldb in #10805
- Add Azure gpt-4o-mini-tts by @emerzon in #10807
- Add Azure Cohere Embed v4 by @emerzon in #10806
- [Fix] Allow overriding all constants using a .env variable by @ishaan-jaff in #10803
- [Bug Fix] Presidio integration failing and making inference return 500 by @ishaan-jaff in #10197
- [Feat] Allow specifying PII Entities Config when using Presidio Guardrails by @ishaan-jaff in #10810
- [Bedrock] Fixes to only allow accepted fields for tool json schema by @ishaan-jaff in #10062
- Fix: Separate Sync/Async Azure Client Cache to Prevent Type Mismatches by @daikeren in #10781
- fix(litellm_logging.py): log custom headers in requester metadata by @krrishdholakia in #10818
- Migrate cohere to
/v2/embed- adds support for output_dimensions param by @krrishdholakia in #10809 - add support to parse metadata by @jugaldb in #10832
- Team id + Status filter on logs by @NANDINI-star in #10831
- Update Groq models on model_prices_and_context_window.json by @naliotopier in #10077
- [Feat] Add endpoints for adding, deleting, editing guardrails in DB by @ishaan-jaff in #10833
- [Feat UI] Add Guardrails on LiteLLM UI by @ishaan-jaff in #10834
- [UI] Fixes for bulk invite users - add validation for bad inputs by @ishaan-jaff in #10836
- Support OTLP standard environment variables for configuration by @damienpontifex in #10813
- [Feat] Presidio Improvements - Allow adding presidio api base on UI, Test presidio on Test Key, fixes for running presidio hook by @ishaan-jaff in #10840
- patch: validate_team_org_change by @ishaan-jaff in #10835
- Feat add retention config by @jugaldb in #10815
- fix failing test typo by @jugaldb in #10843
- Capture Gemini reasoning tokens usage in streaming mode by @Dima-Mediator in #10789
- Anthropic unified web search + tool cost tracking support by @krrishdholakia in #10846
- avoid error
AttributeError: 'Message' object has no attribute 'audio'by @igtm in #10814 - fix: bedrock-sonnet-caching-cost by @shibaboy in #10752
- Fix/issue 10113 embeddings use non default tokenizer by @camfarineau in #10629
- Add
models importcommand by @msabramo in #10581 - Add documentation on in-memory and disk caching by @user202729 in #10522
- Bedrock pixtral fix by @ajac-zero in #10439
- added fix to resolve delete API by @jugaldb in #10845
- build(deps): bump undici from 6.21.1 to 6.21.3 in /docs/my-website by @dependabot in #10855
- [Refactor] Make Pagerduty a free feature by @ishaan-jaff in #10857
- [Refactor] Move enterprise_routes within litellm_enterprise by @ishaan-jaff in #10860
- refactor: make send_management_endpoint_alert a free feature by @ishaan-jaff in #10863
- [Feat - Guardrails] Expose /apply_guardrail endpoint for directly calling guardrail by @ishaan-jaff in #10867
- [CI/CD] Test: fix mapped tests by @ishaan-jaff in #10870
- [Feat] UI: Add support for Deleting Guardrail on UI by @ishaan-jaff in #10869
- [UI] Allow adding Bedrock, Presidio, Lakera, AIM guardrails on UI by @ishaan-jaff in #10874
- fix(proxy/_types.py): add missing comma for
/v2/rerankby @krrishdholakia in #10861 - fix(streaming_handler.py): patch vertex ai meta llama streaming error by @krrishdholakia in #10878
- build(dependencies)📦: Add numpydoc to dependencies in pyproject.toml by @ericmjl in #10736
- [Feat]: Configure LiteLLM to Parse User Headers from Open Web UI by @damgem in #9802
- Feat/support anonymize in aim guardrail by @hxdror in #10757
- Gemini Multimodal Live API support by @krrishdholakia in #10841
- Feat log retention cleanup by @jugaldb in #10872
- Model filter added on logs tab. (#10877) by @ishaan-jaff in #10891
- fix: fix novita ai docs by @jasonhp in #10882
- [Feat] Add Tracing for guardrails in StandardLoggingPayload, Langfuse by @ishaan-jaff in #10890
- [UI] - Add Guardrail Tracing to LiteLLM SpendLogs by @ishaan-jaff in #10893
- Fix example commands in reliability.md by @wwwillchen in #10894
- Feat: add guardrail tracing to OTEL, Arize phoenix by @ishaan-jaff in #10896
- Key hash filter added by @NANDINI-star in #10903
- [Feat] UI: Allow clicking into a guardrail by @ishaan-jaff in #10904
- [Feat] Add Lakera v2 Guardrail Support by @ishaan-jaff in #10880
- Key alias filter on logs by @NANDINI-star in #10905
- [Feat] handle multi-pod deployment for SpendLogs Retention by @jugaldb in #10895
- [UI] Fixes for using Lakera Guardrail on LiteLLM UI by @ishaan-jaff in #10906
- add constant to config settings.md by @jugaldb in #10908
- Remove extra test file by @jugaldb in #10889
- [UI] Allow editing guardrails by @ishaan-jaff in #10907
- Handle openai gpt file data + add openai 'supports_pdf_input' to all vision models + Support bedrock tool cache pointing by @krrishdholakia in #10897
New Contributors
- @imdigitalashish made their first contribution in #10617
- @LouisShark made their first contribution in #10688
- @OscarSavNS made their first contribution in #10764
- @arizedatngo made their first contribution in #10654
- @jugaldb made their first contribution in https://github.com/BerriAI/litellm/pull/...
Contributors
msabramo, shibaboy, and 25 other contributors
Assets 2
v1.70.0-nightly
What's Changed
- Capture Gemini reasoning tokens usage in streaming mode by @Dima-Mediator in #10789
- Anthropic unified web search + tool cost tracking support by @krrishdholakia in #10846
- avoid error
AttributeError: 'Message' object has no attribute 'audio'by @igtm in #10814 - fix: bedrock-sonnet-caching-cost by @shibaboy in #10752
- Fix/issue 10113 embeddings use non default tokenizer by @camfarineau in #10629
- Add
models importcommand by @msabramo in #10581 - Add documentation on in-memory and disk caching by @user202729 in #10522
- Bedrock pixtral fix by @ajac-zero in #10439
- added fix to resolve delete API by @jugaldb in #10845
- build(deps): bump undici from 6.21.1 to 6.21.3 in /docs/my-website by @dependabot in #10855
- [Refactor] Make Pagerduty a free feature by @ishaan-jaff in #10857
- [Refactor] Move enterprise_routes within litellm_enterprise by @ishaan-jaff in #10860
- refactor: make send_management_endpoint_alert a free feature by @ishaan-jaff in #10863
- [Feat - Guardrails] Expose /apply_guardrail endpoint for directly calling guardrail by @ishaan-jaff in #10867
- [CI/CD] Test: fix mapped tests by @ishaan-jaff in #10870
- [Feat] UI: Add support for Deleting Guardrail on UI by @ishaan-jaff in #10869
- [UI] Allow adding Bedrock, Presidio, Lakera, AIM guardrails on UI by @ishaan-jaff in #10874
- fix(proxy/_types.py): add missing comma for
/v2/rerankby @krrishdholakia in #10861 - fix(streaming_handler.py): patch vertex ai meta llama streaming error by @krrishdholakia in #10878
- build(dependencies)📦: Add numpydoc to dependencies in pyproject.toml by @ericmjl in #10736
- [Feat]: Configure LiteLLM to Parse User Headers from Open Web UI by @damgem in #9802
- Feat/support anonymize in aim guardrail by @hxdror in #10757
- Gemini Multimodal Live API support by @krrishdholakia in #10841
- Feat log retention cleanup by @jugaldb in #10872
- Model filter added on logs tab. (#10877) by @ishaan-jaff in #10891
- fix: fix novita ai docs by @jasonhp in #10882
- [Feat] Add Tracing for guardrails in StandardLoggingPayload, Langfuse by @ishaan-jaff in #10890
- [UI] - Add Guardrail Tracing to LiteLLM SpendLogs by @ishaan-jaff in #10893
- Fix example commands in reliability.md by @wwwillchen in #10894
- Feat: add guardrail tracing to OTEL, Arize phoenix by @ishaan-jaff in #10896
- Key hash filter added by @NANDINI-star in #10903
- [Feat] UI: Allow clicking into a guardrail by @ishaan-jaff in #10904
- [Feat] Add Lakera v2 Guardrail Support by @ishaan-jaff in #10880
- Key alias filter on logs by @NANDINI-star in #10905
- [Feat] handle multi-pod deployment for SpendLogs Retention by @jugaldb in #10895
- [UI] Fixes for using Lakera Guardrail on LiteLLM UI by @ishaan-jaff in #10906
- add constant to config settings.md by @jugaldb in #10908
- Remove extra test file by @jugaldb in #10889
- [UI] Allow editing guardrails by @ishaan-jaff in #10907
New Contributors
- @Dima-Mediator made their first contribution in #10789
- @igtm made their first contribution in #10814
- @shibaboy made their first contribution in #10752
- @camfarineau made their first contribution in #10629
- @ajac-zero made their first contribution in #10439
- @damgem made their first contribution in #9802
- @hxdror made their first contribution in #10757
- @wwwillchen made their first contribution in #10894
Full Changelog: v1.69.3-nightly...v1.70.0-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.70.0-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 190.0 | 206.64782881083894 | 6.289821899591554 | 0.0 | 1882 | 0 | 171.67403099995227 | 1154.6766310000294 |
| Aggregated | Passed ✅ | 190.0 | 206.64782881083894 | 6.289821899591554 | 0.0 | 1882 | 0 | 171.67403099995227 | 1154.6766310000294 |
Contributors
msabramo, shibaboy, and 15 other contributors
Assets 4
v1.69.3-nightly
What's Changed
- fix: URL encode OTEL_EXPORTER_OTLP_TRACES_HEADERS for Phoenix Integration by @arizedatngo in #10654
- [README] Add Llama API in supported providers table by @seyeong-han in #10802
- [Fix] Maximum Vertex Recursion Depth Reached when transforming response_format by @ishaan-jaff in #10798
- add ollama api base defaults by @jugaldb in #10805
- Add Azure gpt-4o-mini-tts by @emerzon in #10807
- Add Azure Cohere Embed v4 by @emerzon in #10806
- [Fix] Allow overriding all constants using a .env variable by @ishaan-jaff in #10803
- [Bug Fix] Presidio integration failing and making inference return 500 by @ishaan-jaff in #10197
- [Feat] Allow specifying PII Entities Config when using Presidio Guardrails by @ishaan-jaff in #10810
- [Bedrock] Fixes to only allow accepted fields for tool json schema by @ishaan-jaff in #10062
- Fix: Separate Sync/Async Azure Client Cache to Prevent Type Mismatches by @daikeren in #10781
- fix(litellm_logging.py): log custom headers in requester metadata by @krrishdholakia in #10818
- Migrate cohere to
/v2/embed- adds support for output_dimensions param by @krrishdholakia in #10809 - add support to parse metadata by @jugaldb in #10832
- Team id + Status filter on logs by @NANDINI-star in #10831
- Update Groq models on model_prices_and_context_window.json by @naliotopier in #10077
- [Feat] Add endpoints for adding, deleting, editing guardrails in DB by @ishaan-jaff in #10833
- [Feat UI] Add Guardrails on LiteLLM UI by @ishaan-jaff in #10834
- [UI] Fixes for bulk invite users - add validation for bad inputs by @ishaan-jaff in #10836
- Support OTLP standard environment variables for configuration by @damienpontifex in #10813
- [Feat] Presidio Improvements - Allow adding presidio api base on UI, Test presidio on Test Key, fixes for running presidio hook by @ishaan-jaff in #10840
- patch: validate_team_org_change by @ishaan-jaff in #10835
- Feat add retention config by @jugaldb in #10815
- fix failing test typo by @jugaldb in #10843
New Contributors
- @arizedatngo made their first contribution in #10654
- @jugaldb made their first contribution in #10805
- @daikeren made their first contribution in #10781
- @naliotopier made their first contribution in #10077
- @damienpontifex made their first contribution in #10813
Full Changelog: v1.69.2-nightly...v1.69.3-nightly
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.69.3-nightly
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 271.38705055312096 | 6.10229777560623 | 0.0 | 1826 | 0 | 225.92644499991366 | 2090.630247999968 |
| Aggregated | Passed ✅ | 250.0 | 271.38705055312096 | 6.10229777560623 | 0.0 | 1826 | 0 | 225.92644499991366 | 2090.630247999968 |
Contributors
daikeren, damienpontifex, and 8 other contributors
Assets 4
v1.69.0.patch1-stable
Full Changelog: v1.69.0-stable...v1.69.0.patch1-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_release_branch-v1.69.0.patch1-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 210.0 | 225.79693725160666 | 6.241896325571918 | 0.0 | 1868 | 0 | 189.1762820000622 | 1291.5362580000078 |
| Aggregated | Passed ✅ | 210.0 | 225.79693725160666 | 6.241896325571918 | 0.0 | 1868 | 0 | 189.1762820000622 | 1291.5362580000078 |