domains / families / data / vaex

vaex

combined

Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient visualization of big data, and machine learning on large datasets. Apply when users need to work with large CSV/HDF5/Arrow/Parquet files, perform fast statistics on massive datasets, create visualizations of big data, or build ML pipelines that do not fit in memory. Processes large tabular scientific datasets with Vaex expressions, filtered views, streamed statistics, binned visualizations, and file conversion. Use for larger-than-RAM HDF5, Arrow, CSV, or Parquet analysis, virtual feature engineering, or Vaex ML preprocessing; distinguishes these operations from estimators and conversions that materialize data.

When to use

• Processing tabular datasets larger than available RAM (gigabytes to terabytes)
• Performing fast statistical aggregations on massive datasets
• Creating visualizations and heatmaps of large datasets
• Building machine learning pipelines on big data
• Converting between data formats (CSV, HDF5, Arrow, Parquet)
• Needing lazy evaluation and virtual columns to avoid memory overhead
• Working with astronomical data, financial time series, or other large-scale scientific datasets
• Use Vaex for columnar analysis on a single machine when data exceeds RAM, especially repeated reductions and histograms over local Vaex HDF5 or Arrow files. Expressions and virtual columns defer computation; reductions normally execute imm…

Skill page

Conflict

The K-Dense-AI copy is the more rigorous, verified procedure: it pins versions, defines a 7-step workflow, includes a small executable example, and explicitly warns that out-of-core storage does not make every operation memory bounded. The Vibe-Skills and OpenClaw-Medical-Skills copies are near-identical, more promotional summaries that add a Quick Start pattern, a 'Working with References' routing section, and best-practice bullets, but they omit the version constraints, the executable validation example, and the memory-bound caveats. Because the K-Dense copy lacks the explicit reference-routing guidance and the other two lack the verified workflow and constraints, crowning one would hide a real alternative.