domains / families / research / sigmetrics-experiments

sigmetrics-experiments

separate

Use when designing or auditing ACM SIGMETRICS evaluations, covering theorem-plus-validation rigor, stating and testing modeling assumptions, analysis-vs-simulation agreement, real workloads and traces, fairly tuned baselines, statistics and confidence intervals for stochastic systems, learning guarantees, measurement provenance, and matching evidence to the shape of each performance claim.

When to use

• designing ACM SIGMETRICS evaluations
• auditing ACM SIGMETRICS evaluations
• theorem-plus-validation rigor
• stating and testing modeling assumptions
• analysis-vs-simulation agreement
• real workloads and traces
• fairly tuned baselines
• statistics and confidence intervals for stochastic systems

When not to use

• leaderboard numbers alone
• benchmark score with no guarantee

Skill page

Source