domains / families / devops / model-serving-kubernetes

model-serving-kubernetes

authentic

Deploy ML models on Kubernetes with KServe (formerly KFServing) and NVIDIA Triton Inference Server. Includes canary deployments, autoscaling, model versioning, A/B testing, and GPU resource management for production model serving.

When to use

• Serving scikit-learn, PyTorch, TensorFlow, or ONNX models at scale
• Implementing canary deployments and A/B testing for ML models
• Autoscaling inference pods based on request rate or GPU metrics
• Deploying LLMs with Triton or KServe on Kubernetes
• Managing multiple model versions with traffic splitting

Skill page

Source