domains / families / research / model-merging

model-merging

separate

Merge multiple fine-tuned models using mergekit to combine capabilities without retraining. Use when creating specialized models by blending domain-specific expertise (math + coding + chat), improving performance beyond single models, or experimenting rapidly with model variants. Covers SLERP, TIES-Merging, DARE, Task Arithmetic, linear merging, and production deployment strategies.

When to use

• **Combine capabilities** from multiple fine-tuned models without retraining
• **Create specialized models** by blending domain-specific expertise (math + coding + chat)
• **Improve performance** beyond single models (often +5-10% on benchmarks)
• **Reduce training costs** - no GPUs needed, merges run on CPU
• **Experiment rapidly** - create new model variants in minutes, not days
• **Preserve multiple skills** - merge without catastrophic forgetting

Skill page

Source