domains / science / variant-calling-filtering / filtering-best-practices

filtering-best-practices

Skill card

combined

The two skills cover the same domain but have distinct, non-overlapping material. The first copy (OpenClaw) provides a concise decision tree and a comprehensive reference for GATK hard filter thresholds and bcftools expression syntax. The second copy (GPTomics) provides a deeper methodological framework, including VQSR mechanics, VETS, NVScoreVariants, and specific warnings about version compatibility and the 'missing-annotation-passes' rule. Neither is a simple reproduction of the other; the first is a 'how-to' reference, while the second is a 'why and when' guide with more advanced tooling. Combining them provides a complete skill that covers both the basic execution and the advanced decision-making logic.

Repositories

Other copies

Combined skill

# Variant Filtering Best Practices

## 1. Method Selection

**Governing Principle:** Site-level filtering (is the site real?) and genotype-level filtering (is the genotype trustworthy?) are orthogonal. SNPs and indels must be filtered separately.

**Decision Tree:**
1. **Somatic Data:** Use GATK `FilterMutectCalls`. Do NOT use VQSR or germline hard filters.
2. **Germline Data:**
   - **Large Cohort (Human, >30 exomes or WGS):** Use VQSR or VETS (BETA). 
     - *Note:* VQSR requires truth sets (HapMap/Omni/Mills). Do not use DP as an annotation for exomes.
   - **Small Cohort (Single exome/panel):** 
     - *Preferred:* NVScoreVariants (if GPU available) or VETS.
     - *Fallback:* GATK Hard Filters (see thresholds below).
   - **Non-Model Organism:** Use Hard Filters.

## 2. GATK Hard Filter Thresholds

Use `gatk VariantFiltration` for small cohorts or when ML tools are unavailable.

**SNPs:**
```bash
gatk VariantFiltration \
    -R reference.fa \
    -V raw_snps.vcf \
    -O filtered_snps.vcf \
    --filter-expression "QD < 2.0" --filter-name "QD2" \
    --filter-expression "FS > 60.0" --filter-name "FS60" \
    --filter-expression "MQ < 40.0" --filter-name "MQ40" \
    --filter-expression "MQRankSum < -12.5" --filter-name "MQRankSum-12.5" \
    --filter-expression "ReadPosRankSum < -8.0" --filter-name "ReadPosRankSum-8" \
    --filter-expression "SOR > 3.0" --filter-name "SOR3"
```

**Indels:**
```bash
gatk VariantFiltration \
    -R reference.fa \
    -V raw_indels.vcf \
    -O filtered_indels.vcf \
    --filter-expression "QD < 2.0" --filter-name "QD2" \
    --filter-expression "FS > 200.0" --filter-name "FS200" \
    --filter-expression "ReadPosRankSum < -20.0" --filter-name "ReadPosRankSum-20" \
    --filter-expression "SOR > 10.0" --filter-name "SOR10"
```

## 3. bcftools Filtering

Use `bcftools filter` for custom expressions or when GATK is not available.

**Hard Filter (Remove):**
```bash
bcftools filter -e 'QUAL<30' input.vcf.gz -o filtered.vcf
```

**Soft Filter (Mark):**
```bash
bcftools filter -s 'LowQual' -e 'QUAL<30' input.vcf.gz -o marked.vcf
```

**Common Expressions:**
- Basic: `QUAL>30 && DP>10`
- Genotype-level: `FMT/DP>10 && FMT/GQ>20`
- Allele Balance: `GT="het" -> (AD[1]/(AD[0]+AD[1]) > 0.2 && AD[1]/(AD[0]+AD[1]) < 0.8)`

## 4. Quality Metrics Reference

| Metric | Meaning | Good Value (SNP) | Good Value (Indel) |
|--------|---------|------------------|--------------------|
| QD | Quality by Depth | >2 | >2 |
| FS | Fisher Strand | <60 | <200 |
| MQ | Mapping Quality | >40 | >40 |
| MQRankSum | MQ Rank Sum | >-12.5 | >-20 |
| ReadPosRankSum | Read Position | >-8 | >-20 |
| SOR | Strand Odds Ratio | <3 | <10 |

## 5. Version Compatibility

- **GATK:** 4.6+ (Note: CNNScoreVariants is deprecated; use NVScoreVariants).
- **bcftools:** 1.19+
- **cyvcf2:** 0.30+

Always verify installed versions with `gatk --list` and `bcftools --version` before scripting.