domains / families / data / paidf-curation-and-retrieval

paidf-curation-and-retrieval

separate

Use when operating PAIDF Curation and Retrieval or NVIDIA Cosmos Curator pipelines (split, filter, caption, embed, dedup, shard, image annotate) or PAIDF Data Mining nearest-neighbor matching on Curator embeddings. Activate for Make or CLI pipeline config, GPU run preflight, FFmpeg sidecar, SAM3 keys, or Curator-to-TAO handoff. Do not use for generic ETL, vector-database RAG, model training, orchestration, or embeddings outside Cosmos Curator and PAIDF Data Mining.

When to use

• Turn raw video and image collections into curated, training-ready datasets. This skill configures and runs cosmos-curator pipelines (clip splitting, filtering, captioning, embeddings, SAM3 event verification, dedup, WebDataset sharding, im…

When not to use

• Requires NVIDIA GPU(s); pipelines are not CPU-only. SHM is bounded by host RAM.
• Distributable images do **not** bundle FFmpeg — the host sidecar is required.
• Emit configs as flat `snake_case` YAML. PAIDF v1.1 validation also accepts
• Upstream image annotate supports config-file mode via
• The full pitfall list (build, config, image pipeline, SAM3 keys) is in
• Dataset Search (CDS and Milvus compose) is a separate Make surface

Skill page

Source