domains / families / devops / agent-platform-eval-flywheel

agent-platform-eval-flywheel

separate

Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology. Use when generating synthetic user scenarios, evaluating an agent or model, building an eval dataset, picking or writing evaluation metrics, analyzing failures, comparing results before and after a fix, or when guidance is needed on Agent Platform eval methodology — including dataset schema, LLM-as-judge scoring, and common failure causes. For fine-tuning, use agent-platform-tuning. For general production deployment, use agent-platform-deploy.

When to use

• Evaluating GenAI agents or models with the Agent Platform GenAI Evaluation
• Creating evaluation datasets from session traces, pandas DataFrames, or
• Selecting, configuring, or writing custom evaluation metrics.
• Analyzing rubric verdicts, loss patterns, and clustering failures.
• Suggesting concrete code/prompt improvements based on eval results.
• Evaluating a model served on an Agent Platform **endpoint** (BYOM) or a

When not to use

• Fine-tuning
• General production deployment

Skill page

Source