families / data
data
ai-evaluation-and-leaderboards
A set of skills for creating, running, and analyzing AI model evaluations and leaderboards.
These skills focus specifically on the evaluation, benchmarking, and ranking of AI models, distinct from general AI project development or data querying.
- Set up an evaluation pipeline to benchmark a new LLM against existing models.
- Generate a leaderboard comparing the performance of different AI models on a specific dataset.
azure-data-services
A collection of skills for interacting with Azure data services including search, storage, and database management across multiple programming languages.
The skills differ by the specific Azure service (Search, Storage, Postgres, Maps) and the programming language (Python, TypeScript, .NET) used to interact with them.
- Index a collection of PDF documents into Azure Cognitive Search using Python.
- Upload a large dataset to Azure Data Lake Storage Gen2 using the Python SDK.
ncbi-bioinformatics-access
A suite of skills for querying, retrieving, and analyzing biological data from NCBI resources including GenBank, GEO, SRA, and Entrez databases.
These skills focus specifically on accessing and retrieving data from NCBI-hosted databases and tools, distinguishing them from local analysis tools like BLAST or visualization methods like volcano plots.
- Search for and download raw sequencing data from the SRA database for a specific organism.
- Retrieve gene expression data from GEO and link it to relevant protein sequences in UniProt.
clinical-asr-pipeline
A workflow for building, configuring, and evaluating automatic speech recognition systems specifically tailored for clinical data.
These skills focus on the end-to-end lifecycle of speech recognition models for clinical audio, whereas the clinical-data-cleaner is used for general data hygiene and does not handle audio processing or model training.
- Set up the environment and build the initial ASR model for processing doctor-patient audio recordings.
- Evaluate the accuracy of the fine-tuned clinical ASR model against a labeled dataset of medical transcripts.
data-visualization-and-analysis
A family of skills focused on transforming, analyzing, and visualizing datasets using specialized libraries and tools.
These skills differ from general data engineering or storage skills by focusing specifically on the analytical and visual representation of data rather than its ingestion, synchronization, or raw storage management.
- Create an interactive dashboard using pandas and plotly to visualize sales trends.
- Profile a large CSV file to identify data quality issues and generate summary statistics.
database-schema-design
Designing and structuring database schemas, including normalization, indexing strategies, and entity relationships for SQL and NoSQL systems.
This skill focuses on the structural design and modeling of data (schemas, ERDs, normalization) rather than runtime performance tuning, specific vendor administration, or data extraction.
- Design a normalized schema for an e-commerce platform with users, products, and orders.
- Create an entity-relationship diagram for a hospital management system and suggest appropriate indexes.
dataset-management
A collection of skills for locating, converting, splitting, documenting, and accessing machine learning datasets.
These skills differ by their specific role in the dataset lifecycle, ranging from discovery and format conversion to partitioning, documentation, and platform-specific access.
- Find a dataset for sentiment analysis and convert it to JSONL format.
- Split the CIFAR-10 dataset into train and test sets and generate a datasheet.
differential-expression-analysis
A workflow for identifying statistically significant changes in gene expression using specialized statistical frameworks and visualizing the resulting data.
The skills differ by their specific function in the pipeline, ranging from foundational tool usage (deseq2-basics, edger-basics) and specialized data types (timeseries-de) to the processing of outputs (de-results) and their graphical representation (de-visualization).
- Run a DESeq2 analysis on my RNA-seq count matrix and generate a volcano plot of the top hits.
- Compare time-series expression data using edgeR to identify genes with significant temporal trends.
distributed-training-orchestration
A family of skills for configuring and executing large-scale model training across multiple devices or nodes using various parallelism strategies.
These skills differ in their underlying frameworks and specific parallelism implementations, such as using PyTorch's native FSDP versus Ray's actor-based orchestration or DeepSpeed's ZeRO optimization.
- Configure a PyTorch model to use Fully Sharded Data Parallelism (FSDP) for efficient memory usage on a multi-GPU cluster.
- Set up a distributed training job using Ray Train to scale a model across multiple nodes with automatic resource management.
excel-data-analysis-and-formatting
A suite of skills for performing statistical analysis, data cleaning, and visual formatting on Excel spreadsheets.
The skills differ by their primary function, ranging from data preparation (cleaning, deduplication) to analytical modeling (pivot tables, time-series) and visual presentation (coloring, data bars).
- Clean the sales dataset by removing duplicates and handling missing values, then create a pivot table to summarize revenue by region.
- Apply conditional formatting to highlight outliers and top performers in the performance sheet, and add data bars to visualize the KPI metrics.
financial-data-sourcing
A collection of skills for retrieving, analyzing, and processing financial market data from various providers and APIs.
These skills differ by the specific data provider (e.g., Tushare, Yahoo, iFinD) or the specific analytical focus (e.g., historical cost vs. general collection) they utilize.
- Fetch the last 5 years of daily stock prices for Apple using the Yahoo Finance API.
- Retrieve historical financial statements for a specific company using Tushare data.
data-experimentation-and-pipeline-orchestration
A family of skills covering the design and analysis of statistical experiments alongside the authoring and management of data pipeline workflows.
The listed skills differ by focusing on either the statistical methodology of product experiments or the technical infrastructure of data pipeline orchestration.
- Design an A/B test for a new feature and calculate the required sample size.
- Create a production Airflow DAG to orchestrate a daily data batch job.
geospatial-data-workflows
A collection of skills for processing, analyzing, and visualizing geographic data using Python libraries and schema definitions.
These skills differ by their specific focus on either the Python implementation details (geopandas), data structure definitions (geo-schema), analytical methods (geospatial-analysis), visualization standards (geospatial-viz-guide), or specific routing data handling (geospatial-routing-data).
- How do I perform a spatial join between two GeoDataFrames using GeoPandas?
- What is the recommended schema structure for storing routing data in a geospatial database?
image-data-processing
A set of skills for extracting, analyzing, and augmenting data from images using AI models.
These skills differ by their specific output format, ranging from structured data extraction and attribute augmentation to vector embeddings and spatial grounding.
- Extract structured data from a scanned invoice image.
- Generate bounding box coordinates for all cars in a street view photo.
llm-inference-and-evaluation
A collection of skills for deploying high-performance LLM inference engines and building tools to validate and review model outputs.
The inference skills (sglang, tensorrt-llm, vllm) focus on serving and optimizing model execution, while the evaluation skills (build-review-interface, validate-evaluator) focus on assessing output quality and alignment.
- Deploy a vLLM server to serve a Llama-3 model with continuous batching.
- Build a browser-based interface to review LLM traces and calibrate an LLM judge against human labels.
metric-engineering
A suite of skills for defining, calculating, structuring, and validating business metrics within a data environment.
These skills differ by their specific role in the metric lifecycle, ranging from structural definition and semantic mapping to mathematical calculation, causal attribution, and integrity validation.
- Build a hierarchical metric tree for our revenue dashboard and define the semantic layer for each node.
- Calculate the month-over-month churn rate and use attribution analysis to identify which product features drove the change.
ml-pipeline-orchestration
A family of skills focused on designing, building, and managing end-to-end machine learning pipelines and MLOps workflows.
These skills differ from general ML engineering or specific component setup by focusing specifically on the orchestration, workflow management, and end-to-end pipeline structure rather than individual model tuning or isolated infrastructure tasks.
- Set up a complete MLOps pipeline that handles data preparation, model training, and deployment.
- Orchestrate a prediction stack workflow to ensure seamless integration from feature engineering to model serving.
model-quantization
Techniques for reducing the memory footprint and inference latency of machine learning models by lowering their numerical precision.
These skills focus specifically on post-training compression and precision reduction methods, distinct from training, fine-tuning, or general inference serving.
- Convert a large language model to GGUF format for local deployment.
- Apply AWQ quantization to a transformer model to fit it within a 16GB GPU.
ontology-management
A suite of tools for exploring, validating, mapping, and managing domain-specific ontologies, including NVIDIA-specific configurations and general material science mappings.
The skills differ by their specific function within the ontology lifecycle, ranging from general exploration and validation to specific NVIDIA environment setup and material science term mapping.
- Map the crystal structure description of a new sample to the appropriate ontology classes and properties.
- Validate the consistency of the current ontology registry and check for any missing definitions.
cuopt-optimization-workflow
A suite of skills for installing, formulating, and solving optimization problems using NVIDIA cuOpt and related solvers.
These skills collectively cover the end-to-end process of setting up the cuOpt environment, defining optimization models, and executing solutions via specific APIs or solver workflows.
- Install the cuOpt library and set up the Python environment for numerical optimization.
- Formulate a mixed-integer linear programming problem and solve it using the cuOpt server API.
portaljs-portal-management
A suite of skills for creating, configuring, deploying, and migrating ArcGIS Portal for JavaScript applications.
These skills differ by their specific function in the PortalJS lifecycle, ranging from initial project creation and component addition to deployment and migration.
- Create a new PortalJS project and add a map and chart to the dashboard.
- Migrate an existing ArcGIS application to PortalJS and deploy it to the production environment.
data-process-analysis
A suite of skills for examining and diagnosing various aspects of data processing workflows, including performance, drivers, objects, variants, and specific process issues.
These skills differ by their specific focus area within the data process, ranging from high-level performance metrics and causal drivers to structural object analysis, variant comparison, and deep-dive process investigation.
- Why is the nightly ETL job taking longer than usual?
- Identify the root cause of the data corruption in the user profile pipeline.
rag-pipeline-development
A family of skills for designing, building, optimizing, and evaluating Retrieval-Augmented Generation systems.
These skills differ by their specific focus within the RAG lifecycle, ranging from architectural design and implementation to performance optimization and quality evaluation.
- Design a hybrid search pipeline with reranking for a legal document Q&A system.
- Evaluate the retrieval precision and generation faithfulness of an existing RAG application.
sql-performance-optimization
A set of skills focused on analyzing, tuning, and improving the efficiency and speed of SQL queries and database operations.
These skills specifically target the improvement of query execution speed and resource usage, distinguishing them from general query generation, schema design, or CLI interface tools.
- Analyze the EXPLAIN plan for this slow query and suggest indexing strategies to reduce execution time.
- Refactor this complex aggregation query to improve performance and eliminate unnecessary full table scans.
stata-data-management
A suite of skills for inspecting, cleaning, auditing, and tracking the provenance of datasets within Stata.
These skills focus specifically on data quality, structure, and lineage rather than statistical modeling or visualization.
- Audit the current dataset for missing values and document the source of each variable.
- Inspect the data structure and clean the 'age' column by removing outliers before analysis.
statistical-modeling-and-inference
A family of skills for performing statistical hypothesis testing, fitting regression and time-series models, and generating statistical visualizations using Python libraries like scipy, statsmodels, and seaborn.
These skills focus on the execution of statistical computations, model fitting, and visualization, distinguishing them from advisory or software-specific QA tasks.
- Run a multiple regression analysis on the dataset and provide p-values for each coefficient using statsmodels.
- Create a pair plot to visualize the relationships between variables and check for normality using seaborn.
text-analysis-and-extraction
A collection of tools for parsing, mining, and extracting meaningful insights from unstructured text data.
These skills differ by their specific focus on either semantic representation (sentence-transformers), structural parsing (text-parser), statistical extraction (keyword-extractor), or high-level analytical guidance (text-mining-guide).
- Extract the top 10 keywords from a large corpus of customer support tickets.
- Parse a messy log file and identify sentences containing filler words for quality assessment.
tao-model-training
A collection of skills for training and fine-tuning various machine learning models, including computer vision architectures like RT-DETR and SegFormer, as well as NLP sentence transformers.
The skills differ by the specific model architecture or domain they target, ranging from NVIDIA TAO computer vision models to Hugging Face sentence transformers.
- Fine-tune a sentence-transformers model for semantic search using a custom dataset.
- Train an RT-DETR model for object detection on a custom image dataset.
vector-database-operations
A collection of skills for selecting, configuring, and optimizing vector databases and search algorithms to support semantic retrieval and similarity matching.
These skills differ by focusing on specific vector database vendors (e.g., Milvus, Pinecone, Qdrant) or general engineering patterns and tuning strategies for vector search infrastructure.
- Configure an HNSW index in Qdrant to balance recall and latency for a large-scale recommendation system.
- Compare the performance of Milvus and Pinecone for a RAG application requiring low-latency semantic search.
web-data-extraction
A collection of tools and frameworks for scraping, crawling, and extracting structured data from websites and web archives.
These skills differ by their specific target platforms (e.g., Amazon, Twitter, Substack), underlying frameworks (e.g., Crawlee, Scrapling), or specialized capabilities like archive access and adaptive anti-bot evasion.
- Scrape product prices and reviews from Amazon using the pangolin-amazon-scraper skill.
- Extract text content from a protected webpage using scrapling's anti-bot bypass features.