domains / families / devops / gke-ai-troubleshooting-tpu-metrics-monitoring

gke-ai-troubleshooting-tpu-metrics-monitoring

separate

Monitors and troubleshoots GKE TPU workloads, nodes, and node pools using GKE system metrics and PromQL. Use when monitoring TensorCore duty cycle, TPU memory, node readiness, multi-host TPU node pool availability, host maintenance or preemption interruptions, and calculating MTTR or MTBI metrics for GKE TPUs. Don't use for general non-TPU GKE workload monitoring or non-metric TPU debugging.

When to use

• monitoring TensorCore duty cycle
• monitoring TPU memory
• checking node readiness
• monitoring multi-host TPU node pool availability
• analyzing host maintenance or preemption interruptions
• calculating MTTR or MTBI metrics for GKE TPUs

When not to use

• general non-TPU GKE workload monitoring
• non-metric TPU debugging

Skill page

Source