domains / science / genomic-variant-storage / tiledbvcf

tiledbvcf

Skill card

combined

The K-Dense-AI copy is a highly specific, version-pinned (0.40.3) procedure that includes critical coordinate system warnings (0-based vs 1-based) and specific ABI pins for macOS ARM64. The other two copies (Vibe-Skills and OpenClaw-Medical-Skills) are identical to each other and represent a more generic, older version of the skill that uses deprecated installation flags (CONDA_SUBDIR=osx-64) and lacks the coordinate safety checks. Because the K-Dense copy contains material steps (verification, specific pins) and constraints (coordinate conventions) that the others lack, and the others contain a different (older) installation path, crowning one would hide the alternative or the specific safety constraints.

Repositories

Other copies

Combined skill

---
name: tiledbvcf
description: Stores and retrieves genomic variant calls with TileDB-VCF. Use for indexed single-sample VCF/BCF ingestion, incremental cohorts, region and sample queries, streaming results, allele statistics, QC, and VCF/BCF export locally or through TileDB Cloud.
license: MIT license
metadata:
  version: "1.3"
  skill-author: Jeremy Leipzig
---

# TileDB-VCF

## When to use

Use for cohort variant storage, incremental sample ingestion, interval queries, and exporting subsets for downstream genomics. TileDB-VCF stores records; it does not perform joint variant calling, association testing, normalization, or population structure adjustment.

## Install and verify the native stack

The `tiledbvcf` Python distribution is not published on PyPI. The official `tiledb` Conda channel supplies `tiledbvcf-py`.

**For TileDB-VCF 0.40.3 (Recommended):**
Native Apple Silicon no longer requires forcing `CONDA_SUBDIR=osx-64`. Use Python 3.9-3.12.

```bash
conda create -n tiledb-vcf -c tiledb -c conda-forge \
  python=3.12 tiledbvcf-py=0.40.3 \
  azure-core-cpp=1.16.2 azure-storage-blobs-cpp=12.16.0 \
  azure-storage-files-datalake-cpp=12.14.0 capnproto=1.4.0 c-blosc2=2.23.1
conda activate tiledb-vcf
python -c 'import tiledbvcf; print(tiledbvcf.version)'
```

**For older versions (Legacy):**
If using older builds on M1 Macs, you may need to force the x86 architecture:
```bash
CONDA_SUBDIR=osx-64
conda config --env --set subdir osx-64
conda create -n tiledb-vcf "python<3.10"
mamba install -y -c conda-forge -c bioconda -c tiledb tiledb-py tiledbvcf-py pandas pyarrow numpy
```

## Prepare and create a cohort

1. Record reference assembly, contig naming/lengths, callers, and sample identity.
2. Each input must contain **one sample**, be coordinate sorted, and have an index (`.csi` or `.tbi`).
3. Create the dataset explicitly, then ingest. Opening `mode="w"` alone does not create its schema.

```python
import tiledbvcf

uri = "cohort"
config = {"sm.compute_concurrency_level": "2", "sm.io_concurrency_level": "2"}
with tiledbvcf.Dataset(uri, mode="w", tiledb_config=config) as ds:
    ds.create_dataset(extra_attrs=["fmt_GT", "fmt_DP"])
    ds.ingest_samples(["sample1.vcf.gz"], threads=2, total_memory_budget_mb=512)
```

## Query completely and interpret coordinates correctly

**CRITICAL: Coordinate Conventions**

| Surface | Coordinate convention |
| --- | --- |
| Python `regions=["chr1:10-14"]` | 1-based, closed interval |
| Returned `pos_start`, `pos_end` | 1-based, inclusive record endpoints |
| BED input | 0-based, half-open |
| `read_variant_stats()` `pos` column | **0-based**; add one before joining to VCF POS |

Queries return **overlapping records**, not just records starting inside the interval.

```python
cfg = tiledbvcf.ReadConfig(memory_budget_mb=128, tiledb_config=config)
attrs = ["sample_name", "contig", "pos_start", "pos_end", "alleles", "fmt_GT"]
with tiledbvcf.Dataset(uri, mode="r", tiledb_config=config) as ds:
    df = ds.read(attrs=attrs, regions=["chr1:10-14"], read_config=cfg)
```

## Export

```python
with tiledbvcf.Dataset(uri, mode="r", tiledb_config=config) as ds:
    ds.export(
        regions=["chr21:8220186-8405573"],
        samples=["HG00101"],
        output_format="v",
        output_dir="./export"
    )
```