domains / families / data / rag-perf

rag-perf

separate

Performance benchmarking for a deployed NVIDIA RAG Blueprint server: profiling pass + aiperf load test driven by a single YAML config. Not for accuracy / RAGAS scoring (use rag-eval) or for deploying / repairing services (use rag-blueprint).

When to use

• Drive a deployed NVIDIA RAG Blueprint server with a YAML config, run a server-side **profiling pass** (per-stage timing, citation quality, bottleneck inference) and an optional **aiperf load test** (TTFT / E2E / token & request throughput…
• **Accuracy / RAGAS** scoring of answer quality → use the **rag-eval** skill.
• **Deploying, repairing, or configuring services** (compose, helm, NIM env vars) → use the **rag-blueprint** skill.
• **Production monitoring / alerting** — rag-perf is a one-shot benchmark tool.
• **Runtime requirement:** a deployed RAG server reachable on the network.

When not to use

• The CLI is **config-only**: author or copy YAML to vary a parameter.
• `load.concurrency` / `rag.vdb_top_k` / `rag.reranker_top_k` accept `int | list[int]`; the validator requires unique list values because each value names a unique point dir.
• `input.file` and `input.synthetic` follow an XOR rule — both set fails validation. When neither is set, `synthetic` auto-fills with defaults so a bare config still validates.
• File-based input format is **inferred from extension only** (`.jsonl` or `.csv`); other extensions are rejected.
• Synthetic generation streams each query to disk as it completes (failure-resilient) but **fails fast on the first LLM error** — partial JSONL is preserved. Re-run after fixing the endpoint.
• Reasoning models (Nemotron Omni, Qwen-Reasoning) require `synthetic.disable_thinking: true` (the default). Without it the model exhausts the token budget on chain-of-thought and `content` returns empty — the generator now raises with a cle…
• aiperf-specific knobs outside the YAML surface (request rate distribution, GPU telemetry config, etc.) require editing `AiperfRunner._base_aiperf_cmd` in `scripts/rag-perf/rag_perf/runner.py`.
• Procedural detail lives under **`references/`** to keep this file concise.

Skill page

Source