v1.3.8

What's Changed

• doc: Update shieldgemma doc name by @Sherlock113 in #4999
• fix: tolerate build error when watching for changes by @frostming in #5000
• doc: correct the requirements by @frostming in #5001
• fix: expose errors occurring before the first yield of stream response by @frostming in #5002
• doc: fix endpoint url related typos by @ssheng in #5007
• docs: Add CUDA version notes by @Sherlock113 in #5004
• docs: Add new project examples by @Sherlock113 in #5010
• docs: Clean up examples by @Sherlock113 in #5011
• docs: Update adaptive batching doc by @Sherlock113 in #5012
• docs: Refactor more examples section by @Sherlock113 in #5014
• ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in #5009
• feat: upload setup.sh to the remote dev by @frostming in #5016
• feat: always make an HTTP call when calling self API methods by @frostming in #5017
• feat: gradio integration by @parano in #5008
• Fix a typo by @snowsky in #5018
• docs: Add quotation marks to python versions by @Sherlock113 in #5019

New Contributors

• @snowsky made their first contribution in #5018

Full Changelog: v1.3.7...v1.3.8

v1.3.7

What's Changed

• fix: do not initialize cloud client when building bento by @frostming in #4998

Full Changelog: v1.3.6...v1.3.7

v1.3.6

What's Changed

• doc: include cli commands in secret doc by @ssheng in #4968
• fix: switch back to post-release version scheme by @frostming in #4969
• doc: Add documentation about platform option of bentoml build by @frostming in #4966
• fix: only add bentoml requirement if absent by @frostming in #4973
• fix: null filename causes parsing failure for image input by @frostming in #4976
• doc: add instruction for bentoml.IODescriptor by @frostming in #4975
• chore: update login error message by @xianml in #4952
• chore: remove grpc client by @aarnphm in #4983
• fix: persist trace across internal requests by @frostming in #4982
• feat: expose request id and trace id in request scope by @frostming in #4977
• fix: add secrets in update/apply commands by @xianml in #4985
• feat: development of release 1.4 by @frostming in #4984
• doc: update quick start to use bentoml importing context manager by @frostming in #4986
• fix: make spinner optional for backward compatibility by @frostming in #4987
• fix: don't initialize cloud client until needed by @frostming in #4989
• doc: fix typo in model composition by @ssheng in #4991
• refactor: unify same schema by @frostming in #4988
• fix(framework): export unsloth lazily by @aarnphm in #4992
• fix: compare bento manifest and dependency client by @frostming in #4993
• fix: structure bento manifest by @frostming in #4994
• fix: allow setting timeout for cloud client; hf api; manifest eq by @frostming in #4995
• fix: with_defaults for docker option and show diff report for bento manifest comparison by @frostming in #4996
• docs: Add ShieldGemma example doc by @Sherlock113 in #4997

Full Changelog: v1.3.5...v1.3.6

v1.3.5

What's Changed

• chore(ci): release workflow fix by @frostming in #4962
• fix: on_deployment failed to run for serve_grpc by @frostming in #4963

Full Changelog: v1.3.4...v1.3.5

v1.3.4post1

What's Changed

• chore(ci): release workflow fix by @frostming in #4962

Full Changelog: v1.3.4...v1.3.4post1

v1.3.4

What's Changed

• fix: don't use temporary directory for temp bento copying by @sauyon in #4938
• refactor: path spec ignore pattern checks children files only by @frostming in #4939
• fix: improve exception handling for pynvml by @rlleshi in #4943
• doc: update cloud deploy with yaml guide by @ssheng in #4944
• doc: fix instructions of http testing by @frostming in #4945
• chore(deps): bump protobufjs from 7.2.4 to 7.2.6 in /grpc-client/node by @dependabot in #4647
• chore(deps): bump transformers from 4.30.0 to 4.38.0 in /examples/inference-graph by @dependabot in #4695
• chore(deps): bump h2 from 0.3.24 to 0.3.26 in /grpc-client/rust by @dependabot in #4635
• chore(deps): bump tar from 6.1.11 to 6.2.1 in /grpc-client/node by @dependabot in #4650
• chore(deps): bump mio from 0.8.8 to 0.8.11 in /grpc-client/rust by @dependabot in #4550
• fix: bentoml.serve to support serving new SDK services from a bento by @frostming in #4946
• docs: Let search engine ignore 1.1 doc by @Sherlock113 in #4950
• docs: Update readme example links by @Sherlock113 in #4951
• ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in #4953
• fix: don't pull models if already exists when puling bento by @frostming in #4954
• feat: lock packages for linux by default by @frostming in #4955
• chore(deps): bump actions/download-artifact from 3 to 4.1.7 in /.github/workflows in the github_actions group across 1 directory by @dependabot in #4957
• docs: Add function calling example doc by @Sherlock113 in #4956
• docs: Add a placeholder RAG doc by @Sherlock113 in #4959
• fix: support pydantic 2.9 by @frostming in #4960
• fix: retry for model and bento upload failure by @frostming in #4961
• fix: reload plugin missing file changes by @frostming in #4958

New Contributors

• @rlleshi made their first contribution in #4943

Full Changelog: v1.3.3...v1.3.4

v1.3.3

What's Changed

• fix: restore keras.save_model compatibility with Keras 3 by @rmarquis in #4922
• fix(cli): do not hide output behind a spinner for building bento by @frostming in #4923
• feat: expose client url and sync client by @frostming in #4925
• fix: always preserve str and bytes when serializing by @frostming in #4929
• fix(cloud-client): raise error if multipart push fails by @Haivilo in #4932
• feat: Pass torch kwargs to bentoml.pytorch.load_model by @TokeReines in #4930
• docs: Add function calling example link by @Sherlock113 in #4934
• fix: pydantic/numpy patching causes Cannot interpret 'numpy.float16 | numpy.float32' as a data type by @frostming in #4933
• fix: uploading large files saving to disk instead of memory by @frostming in #4935

New Contributors

• @rmarquis made their first contribution in #4922
• @TokeReines made their first contribution in #4930

Full Changelog: v1.3.2...v1.3.3

v1.3.2

What's Changed

• fix: press text by @xianml in #4898
• ci: pre-commit autoupdate by @pre-commit-ci in #4897
• docs: Update doc images by @Sherlock113 in #4901
• fix: fix resource type by @FogDong in #4904
• fix: direct return response if tracer id is null by @FogDong in #4899
• docs: update azure byoc instructions by @ssheng in #4905
• fix: metrics duration defined with min,max,factor not taking effect by @frostming in #4903
• refactor: store the cloud context into the container instead of passing it around by @frostming in #4907
• docs: Update README.md by @parano in #4906
• docs: Update messages by @Sherlock113 in #4910
• docs: Update index.rst by @Sherlock113 in #4913
• doc: update the HTTP behavior test instruction by @frostming in #4915
• docs: Add metrics doc by @Sherlock113 in #4912
• feat(cli): reload option for start-http-server by @frostming in #4916
• chore: update pdm.lock file witht the lock targets by @frostming in #4911
• fix(build): preserve the index url options in requirements.txt file by @frostming in #4914
• fix: try to fix the timeout test cases by @frostming in #4917
• doc: update CI status badge by @frostming in #4918

Full Changelog: v1.3.1...v1.3.2

v1.3.1

What's Changed

• docs: Add 2 examples by @Sherlock113 in #4872
• docs: update theme to match branding by @ssheng in #4874
• docs: Add task doc by @Sherlock113 in #4873
• fix: set prometheus env before importing service by @frostming in #4875
• docs: Add API docs by @Sherlock113 in #4877
• fix: inject prometheus env from arbiter by @frostming in #4878
• fix: Don't run bentoml.build_bentofile in subprocess by @frostming in #4876
• docs: update azure byoc instructions by @ssheng in #4881
• docs(azure): update azure byoc instructions by @ssheng in #4882
• fix(framework): transformers vs 1.3.0 "TypeError: init() got an unexpected keyword argument 'task'" by @frostming in #4880
• fix: silence git output by @frostming in #4883
• fix: surface config error on bento build by @frostming in #4888
• docs: Refactor tasks doc by @Sherlock113 in #4890
• Docs: Sync the BYOC doc with blog and update BentoCloud guides index page by @Sherlock113 in #4891
• fix: move user setup to base template to make custom base image work by @frostming in #4893
• fix: Cannot define custom duration histogram buckets via @bentoml.service(metrics=...) by @frostming in #4895

Full Changelog: v1.3.0...v1.3.1

v1.3.0

We are excited to announce the release of BentoML 1.3! Following the feedback received since the launch of 1.2 earlier this year, we are introducing a host of new features and enhancements in 1.3. Below are the key highlights of 1.3 and stay tuned for an upcoming blog post, where we'll provide a detailed exploration of the new features and the driving forces behind the development.

Here are some of the important points to note about 1.3:

• 1.3 ensures full backward compatibility, meaning that Bentos built with 1.2 will continue to work seamlessly with this release.
• We remain committed to supporting 1.2. Critical bug fixes and security updates will be backported to the 1.2 branch.
• The BentoML documentation has been updated with examples and guides for 1.3. More guides will be added in the coming weeks.
• BentoCloud supports Bento Deployments from both 1.2 and 1.3 releases of BentoML.

Now, let’s take a look at the major features and enhancements:

🕙 Implemented BentoML task execution

• Introduced the @bentoml.task decorator to set a task endpoint for executing a long-running workload (such as batch processing or video generation).
• Added the .submit() method to both the sync and async clients, which can submit task inputs via the task endpoint and dedicated worker processes constantly monitor task queues for new work to perform.
• Full compatibility with BentoCloud to run Bentos defined with task endpoints.
• See the Services and Clients doc with examples of a Service API by initializing a long running task in the Service constructor, creating clients to call the endpoint, and retrieving task status.

🚀 Optimized the build cache to accelerate the build process

• Enhanced build speed for bentoml build & containerize through pre-installed large packages like torch
• Switch to uv as the installer and resolver, replacing pip

🔨 Supported concurrency-based autoscaling on BentoCloud

• Added the concurrency configuration to the @bentoml.service decorator to set the ideal number of simultaneous requests a Service is designed to handle.
• Added the external_queue configuration to the @bentoml.service decorator to queue excess requests until they can be processed within the defined concurrency limits.
• See the documentation to configure concurrency and external queue.

🔒 Secure data handling with secrets in BentoCloud:

• You can now create and manage credentials, such as HuggingFace tokens and AWS secrets, securely on BentoCloud and easily apply them across multiple Deployments.
• Added secret subcommands to the BentoML CLI for secret management. Run bentoml secret -h to learn more.

🗒️ Added streamed logs for Bento image deployment.

• Easier to troubleshoot build issues and enable faster development iterations

🙏 Thank you for your continued support!

What's Changed

• fix: change forbid extra keys to false for bentocloud by @FogDong in #4866
• feat(dev): 1.3 by @frostming in #4849
• fix: delete cluster and ns if it is first cluster by @FogDong in #4869
• fix: auto login confirm ask logic by @xianml in #4864
• fix: secret default value by @xianml in #4870
• fix: fix typo in error msg by @FogDong in #4871

Full Changelog: v1.2.20...v1.3.0