Skip to content

Commit

Permalink
Update version to 0.14.1 and runtime doc (#435)
Browse files Browse the repository at this point in the history 
* Update version to 0.14.1 and runtime doc

Signed-off-by: Dan Sun <[email protected]>

* Fix wording

Signed-off-by: Dan Sun <[email protected]>

---------

Signed-off-by: Dan Sun <[email protected]>
  • Loading branch information
@yuzisun
yuzisun authored Dec 25, 2024
1 parent 5bdede6 commit 7c66c4b
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 8 deletions.
11 changes: 4 additions & 7 deletions docs/modelserving/v1beta1/serving_runtime.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

KServe provides a simple Kubernetes CRD to enable deploying single or multiple trained models onto model serving runtimes such as [TFServing](https://www.tensorflow.org/tfx/guide/serving),
[TorchServe](https://pytorch.org/serve/server.html), [Triton Inference Server](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs).
In addition [ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) is the Python model serving runtime implemented in KServe itself with prediction v1 protocol,
[MLServer](https://github.com/SeldonIO/MLServer) implements the [prediction v2 protocol](https://github.com/kserve/kserve/tree/master/docs/predict-api/v2) with both REST and gRPC.
For Hugging Face models, KServe provides [Hugging Face Server](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver) for hosting the transformer based models with Open Inference and OpenAI Protocol.
In addition [ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) is the Python model serving runtime implemented in KServe itself with prediction v1 and Open Inference Protocol(v2),
These model serving runtimes are able to provide out-of-the-box model serving, but you could also choose to build your own model server for more complex use case.
KServe provides basic API primitives to allow you easily build custom model serving runtime, you can use other tools like [BentoML](https://docs.bentoml.org/en/latest) to build your custom model serving image.

Expand Down Expand Up @@ -36,19 +36,16 @@ In a given serving runtime directory the pyproject.toml file contains the exact
| Model Serving Runtime | Exported model | HTTP | gRPC | Default Serving Runtime Version | Supported Framework (Major) Version(s) | Examples |
|-------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|-------------|------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------|
| [Custom ModelServer](https://github.com/kserve/kserve/tree/master/python/kserve/kserve) | -- | v1, v2 | v2 | -- | -- | [Custom Model](custom/custom_model/README.md) |
| [LightGBM MLServer](https://mlserver.readthedocs.io/en/latest/runtimes/lightgbm.html) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v2 | v2 | v1.5.0 (MLServer) | 4 | [LightGBM Iris V2](./lightgbm/README.md) |
| [LightGBM ModelServer](https://github.com/kserve/kserve/tree/master/python/lgbserver) | [Saved LightGBM Model](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html#lightgbm.Booster.save_model) | v1, v2 | v2 | v{{ kserve_release_version }} (KServe) | 4 | [LightGBM Iris](./lightgbm/README.md) |
| [MLFlow ModelServer](https://docs.seldon.io/projects/seldon-core/en/latest/servers/mlflow.html) | [Saved MLFlow Model](https://www.mlflow.org/docs/latest/python_api/mlflow.sklearn.html#mlflow.sklearn.save_model) | v2 | v2 | v1.5.0 (MLServer) | 2 | [MLFLow wine-classifier](./mlflow/v2/README.md) |
| [PMML ModelServer](https://github.com/kserve/kserve/tree/master/python/pmmlserver) | [PMML](http://dmg.org/pmml/v4-4-1/GeneralStructure.html) | v1, v2 | v2 | v{{ kserve_release_version }} (KServe) | 3, 4 ([PMML4.4.1](https://github.com/autodeployai/pypmml)) | [SKLearn PMML](./pmml/README.md) |
| [SKLearn MLServer](https://github.com/SeldonIO/MLServer) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v2 | v2 | v1.5.0 (MLServer) | 1 | [SKLearn Iris V2](./sklearn/v2/README.md) |
| [SKLearn ModelServer](https://github.com/kserve/kserve/tree/master/python/sklearnserver) | [Pickled Model](https://scikit-learn.org/stable/modules/model_persistence.html) | v1, v2 | v2 | v{{ kserve_release_version }} (KServe) | 1.5 | [SKLearn Iris](./sklearn/v2/README.md) |
| [TFServing](https://www.tensorflow.org/tfx/guide/serving) | [TensorFlow SavedModel](https://www.tensorflow.org/guide/saved_model) | v1 | *tensorflow | 2.6.2 ([TFServing Versions](https://github.com/tensorflow/serving/releases)) | 2 | [TensorFlow flower](./tensorflow/README.md) |
| [TorchServe](https://pytorch.org/serve/server.html) | [Eager Model/TorchScript](https://pytorch.org/docs/master/generated/torch.save.html) | v1, v2, *torchserve | *torchserve | 0.9.0 (TorchServe) | 2 | [TorchServe mnist](./torchserve/README.md) |
| [Triton Inference Server](https://github.com/triton-inference-server/server) | [TensorFlow,TorchScript,ONNX](https://github.com/triton-inference-server/server/blob/r21.09/docs/model_repository.md) | v2 | v2 | 23.05-py3 (Triton) | 8 (TensoRT), 1, 2 (TensorFlow), 2 (PyTorch), 2 (Triton) [Compatibility Matrix](https://docs.nvidia.com/deeplearning/frameworks/support-matrix/index.html) | [Torchscript cifar](triton/torchscript/README.md) |
| [XGBoost MLServer](https://github.com/SeldonIO/MLServer) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v2 | v2 | v1.5.0 (MLServer) | 2 | [XGBoost Iris V2](./xgboost/README.md) |
| [XGBoost ModelServer](https://github.com/kserve/kserve/tree/master/python/xgbserver) | [Saved Model](https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html) | v1, v2 | v2 | v{{ kserve_release_version }} (KServe) | 2 | [XGBoost Iris](./xgboost/README.md) |
| [HuggingFace ModelServer](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver) | [Saved Model](https://huggingface.co/docs/transformers/v4.39.2/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) / [Huggingface Hub Model_Id](https://huggingface.co/models) | v1, v2 | -- | v{{ kserve_release_version }} (KServe) | 4 ([Transformers](https://pypi.org/project/transformers/4.43.4/)) | -- |
| [HuggingFace VLLM ModelServer](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver) | [Saved Model](https://huggingface.co/docs/transformers/v4.43.4/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) / [Huggingface Hub Model_Id](https://huggingface.co/models) | v2 | -- | v{{ kserve_release_version }} (KServe) | 0 ([Vllm](https://pypi.org/project/vllm/0.6.1.post2/)) | -- |
| [HuggingFace ModelServer](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver) | [Saved Model](https://huggingface.co/docs/transformers/v4.39.2/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) / [Huggingface Hub Model_Id](https://huggingface.co/models) | v1, v2, OpenAI | -- | v{{ kserve_release_version }} (KServe) | 4 ([Transformers](https://pypi.org/project/transformers/4.43.4/)) | -- |
| [HuggingFace VLLM ModelServer](https://github.com/kserve/kserve/tree/master/python/huggingfaceserver) | [Saved Model](https://huggingface.co/docs/transformers/v4.43.4/en/main_classes/model#transformers.PreTrainedModel.save_pretrained) / [Huggingface Hub Model_Id](https://huggingface.co/models) | v2, OpenAI | -- | v{{ kserve_release_version }} (KServe) | 0 ([Vllm](https://pypi.org/project/vllm/0.6.1.post2/)) | -- |



Expand Down
2 changes: 1 addition & 1 deletion mkdocs.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@ plugins:


extra:
kserve_release_version: 0.14.0
kserve_release_version: 0.14.1
social:
- icon: fontawesome/brands/github
link: https://github.com/kserve
Expand Down

0 comments on commit 7c66c4b

Please sign in to comment.