domains / families / data / rag-eval

rag-eval

separate

Filesystem RAG benchmarks: corpus/, train.json, evaluate_rag.py (RAGAS quality). Not for prod monitoring, latency/throughput benchmarking (use rag-perf), or evals outside this repo layout.

When to use

• Guide agents through NVIDIA RAG Blueprint **filesystem** benchmarks: preparing `corpus/` and `train.json`, running `scripts/eval/evaluate_rag.py`, tuning retrieval and generation flags for **quality** comparisons, interpreting RAGAS JSON o…

When not to use

• Do **not** use this skill for: deploying or repairing services (use rag-blueprint); evaluating APIs without the `corpus/` + `train.json` layout; general ML experimentation unrelated to this evaluator; production monitoring/alerting; or lat…
• Evaluator behavior is fixed to the **filesystem contract** and `evaluate_rag.py`; it does not substitute for custom offline judges or non-RAG benchmarks.
• **Vector DB / embedding** choices follow deployed ingestor and RAG env — not overridden by this CLI alone.
• **Scores depend on** retrieval quality, judge model availability, and `NVIDIA_API_KEY`; empty contexts yield partial RAGAS metrics (see references).
• Large procedural detail lives under **`references/`** to keep routing concise; read those files when the user needs step-by-step conversion, full flags, or error tables.

Skill page

Source