domains / families / devops / gke-inference

gke-inference

separate

Deploys and optimizes AI/ML inference workloads on GKE, using GPUs, TPUs, and model servers. Use when deploying GKE inference servers, configuring GKE GPU resources for inference, or deploying LLMs on GKE. Don't use for migrating existing AI workloads to GKE (use google-cloud-solution-guided-gke-ai-migration), GKE RAG with Cloud SQL/AlloyDB (use google-cloud-solution-rag-enterprise-search-gke-sqldb), or batch/HPC (use gke-batch-hpc).

When to use

• Deploy an AI model (Llama, Gemma, Mistral, etc.) to GKE
• Generate optimized Kubernetes manifests for inference
• Select GPU/TPU accelerators for model serving
• Configure autoscaling for LLM inference

When not to use

• Migrating existing AI workloads to GKE
• GKE RAG with Cloud SQL/AlloyDB
• Batch/HPC workloads

Skill page

Source