v1.2.0

🍱 We are excited to share with you that we have released BentoML v1.2, the biggest release since the launch of v1.0. This release includes improvements from all the learning and feedback from our community over the past year. We invite you to read our release blog post for a comprehensive overview of the new features and the motivations behind their development.

Here are a few key points to note before we delve into the new features:

• v1.2 ensures complete backward compatibility, meaning that Bentos built with v1.1 will continue to function seamlessly with this release.
• We remain committed to supporting v1.1. Critical bug fixes and security updates will be backported to the v1.1 branch.
• BentoML documentation has been updated with examples and guides for v1.2. More guides are being added every week.
• BentoCloud is fully equipped to handle deployments from both v1.1 and v1.2 releases of BentoML.

⛏️ Introduced a simplified service SDK to empower developers with greater control and flexibility.

• Simplified the service and API interfaces as Python classes, allowing developers to add custom logic and use third party libraries flexibly with ease.
• Introduced @bentoml.service and @bentoml.api decorators to customize the behaviors of services and APIs.
• Moved configuration from YAML files to the service decorator @bentoml.service next to the class definition.
• See this example demonstrating the flexibility of the service API by initializing a vLLM AsyncEngine in the service constructor and run inference with continuous batching in the service API.

🔭 Revamped IO descriptors with more familiar input and output types.

• Enable use of Pythonic types directly, without the need for additional IO descriptor definitions or decorations.
• Integrated with Pydantic to leverage its robust validation capabilities and wide array of supported types.
• Expanded support to ML and Generative AI specific IO types.

📦 Updated model saving and loading API to be more generic to enable integration with more ML frameworks.

• Allow flexible saving and loading models using the bentoml.models.create API instead of framework specific APIs, e.g. bentoml.pytorch.save_model, bentoml.tensorflow.save_model.

🚚 Streamlined the deployment workflow to allow more rapid development iterations and a faster time to production.

• Enabled direct deployment to production through CLI and Python API from Git projects.

🎨 Improved API development experience with generated web UI and rich Python client.

• All bentos are now accompanied by a custom-generated UI in the BentoCloud Playground, tailored to their API definitions.
• BentoClient offers a Pythonic way to invoke the service endpoint, allowing parameters to be supplied in native Python format, letting the client efficiently handles the necessary serialization while ensuring compatibility and performance.

🎭 We've learned that the best way to showcase what BentoML can do is not through dry, conceptual documentation but through real-world examples. Check out our current list of examples, and we'll continue to publish new ones to the gallery as exciting new models are released.

• BentoVLLM
• BentoControlNet
• BentoSDXLTurbo
• BentoWhisperX
• BentoXTTS
• BentoCLIP

🙏 Thank you for your continued support!

What's Changed

• chore(deps): bump h2 from 0.3.20 to 0.3.24 in /grpc-client/rust by @dependabot in #4434
• fix: Remove trailing character when building bento with API on Windows by @holzweber in #4455
• fix: Replace backslahes by normal slashes, making bentoml pull possible on windows by @holzweber in #4456
• fix(monitoring): Missing f string by @jianshen92 in #4463
• feat: 1.2 staging by @bojiang in #4366
• chore(deps): bump pdm-project/setup-pdm from 3 to 4 by @dependabot in #4457
• fix(client): Convert string to Path if it isn't like a URL by @frostming in #4469
• docs: Add Model Store doc by @Sherlock113 in #4471
• docs: Clean up BentoCloud doc and add get started doc by @Sherlock113 in #4472
• docs: Update the quickstart by @Sherlock113 in #4474

New Contributors

• @holzweber made their first contribution in #4455

Full Changelog: v1.2.0rc1...v1.2.0

v1.2.0rc1

What's Changed

• fix: correct schema for optional type by @frostming in #4459
• fix: client sending request fail if the input is a list of files by @frostming in #4461
• fix: worker number of start-http-server by @bojiang in #4462

Full Changelog: v1.2.0a7...v1.2.0rc1

v1.2.0a7

What's Changed

• feat: allow disabling GPU allocation via env by @frostming in #4453

Full Changelog: v1.2.0a6...v1.2.0a7

v1.2.0a6

What's Changed

• feat: add adaptive batch size histogram by @frostming in #4438
• ci: Use nox as the task runner by @frostming in #4432
• feat(cli): remove or hide unneeded commands by @frostming in #4441
• docs: Add whisperx use case doc by @Sherlock113 in #4436
• docs: Add Docker deployment doc by @Sherlock113 in #4443
• docs: Add io types doc by @Sherlock113 in #4444
• fix: only inline display images by @frostming in #4442
• docs: Update homepage by @Sherlock113 in #4445
• feat: skip access log for infra endpoints by @frostming in #4447
• feat: add schema in bentoInfo by @xianml in #4439
• fix: fix deployment cli log by @FogDong in #4450
• fix: convert generator to async gen before response by @frostming in #4448

Full Changelog: v1.2.0a5...v1.2.0a6

v1.2.0a5

What's Changed

• fix: image encoding issue when the format is not specified by @frostming in #4435

Full Changelog: v1.2.0a4...v1.2.0a5

v1.2.0a4

What's Changed

• fix: make client timeout consistent with service config by @frostming in #4425
• fix: fix env in deploy and create error msg by @FogDong in #4426
• fix: port reservation causes bind error on Windows by @frostming in #4427
• fix: add more status check in wait until ready by @FogDong in #4428
• fix: use spinner log instead of console log by @FogDong in #4429
• fix: add config dict to cli by @FogDong in #4424
• docs: Add client and use case docs by @Sherlock113 in #4430
• fix(sdk): snake case bento names by @bojiang in #4431
• docs: Add SDXL turbo use case by @Sherlock113 in #4433
• docs: Update readme doc by @Sherlock113 in #4409

Full Changelog: v1.2.0a3...v1.2.0a4

v1.2.0a3

What's Changed

• fix(server): start-http-server for v1.0 bentos by @bojiang in #4421
• fix: refactor cli & sdk for deployment by @FogDong in #4419
• fix: fix get bento from cloud by @FogDong in #4423
• fix: change entry_service to optional by @xianml in #4422

Full Changelog: v1.2.0a2...v1.2.0a3

v1.2.0a2

What's Changed

• feat: allow url as file input by @frostming in #4411
• fix(sdk): validate service name; force lowercase by @bojiang in #4414
• refactor(bento): add entry_service to bento info by @bojiang in #4417
• fix: set content type from annotation by @frostming in #4416
• fix(server): include system headers by @bojiang in #4418
• fix: loading service from an absolute path by @frostming in #4415
• docs: Add BLIP, CLIP, and sentence embedding use cases by @Sherlock113 in #4420

Full Changelog: v1.2.0a1...v1.2.0a2

v1.2.0a1

What's Changed

• fix(docs): remove eager type evaluation for python 3.8 on openllm examples by @aarnphm in #4393
• docs: fix typo in tritonserver_type parameter and code example by @ilyahabr in #4394

New Contributors

• @ilyahabr made their first contribution in #4394

Full Changelog: v1.2.0a0...v1.2.0a1

v1.2.0a0

What's Changed

• feat: add preview feature for output by @jinyang1994 in #4319
• feat: add feature for form validation by @jinyang1994 in #4322
• fix: example and bento config by @FogDong in #4324
• fix: set wrong default value when type is array by @jinyang1994 in #4332
• feat: add v2 config and json override env by @FogDong in #4331
• feat: config override by @frostming in #4334
• feat: models.save api and tests by @MingLiangDai in #4307
• fix: fix config migration by @FogDong in #4341
• fix: async api call by @xianml in #4349
• fix(config): typo on override by @Haivilo in #4351
• feat: reorganize the new SDK package by @frostming in #4337
• feat: support .python-version symlink by @aarnphm in #4354
• feat: add loading status when form is submitting by @jinyang1994 in #4361
• feat: add e2e tests for new SDK by @frostming in #4352
• fix(with_config): annotate return type by @aarnphm in #4355
• Chore: add supported gpu type by @xianml in #4363
• fix(config): make sure to escape quotation for migration of services config by @aarnphm in #4369
• fix(sdk): identify async by original func by @bojiang in #4370
• chore(bento.yaml): move fields into services by @bojiang in #4372
• fix: add services in manifest by @FogDong in #4373
• chore(sdk): envs in bentofile by @bojiang in #4378
• chore(cloud): include envs in manifest by @bojiang in #4379
• chore: cherry-pick SSE utils into 1.2 branch by @aarnphm in #4375
• fix: correct 1.2 model list and tag format when pushing bento by @Haivilo in #4381
• feat: Update the bento yaml schema by @frostming in #4371
• ci: pre-commit autoupdate [skip ci] by @pre-commit-ci in #4382
• chore(sdk): able to specify service name by @bojiang in #4377
• feat(bentocloud): deployment v2 api client + cli by @Haivilo in #4335
• chore(deps): bump github/codeql-action from 2 to 3 by @dependabot in #4343
• feat(sdk): use attribute chain as dependency import string by @frostming in #4385
• chore(example): change name to avoid conflict by @bojiang in #4387
• refactor(impl): 1.2 loader by @bojiang in #4388
• fix: refactor deployment v2 client and cli by @FogDong in #4383

Full Changelog: v1.1.11...v1.2.0a0