domains / families / research / peft

peft

separate

Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.

When to use

• Fine-tuning 7B-70B models on consumer GPUs (RTX 4090, A100)
• Need to train <1% parameters (6MB adapters vs 14GB full model)
• Want fast iteration with multiple task-specific adapters
• Deploying multiple fine-tuned variants from one base model
• Fine-tuning 70B models on single 24GB GPU
• Memory is the primary constraint
• Can accept ~5% quality trade-off vs full fine-tuning
• Training small models (<1B parameters)

When not to use

• Training small models (<1B parameters)
• Scenarios requiring maximum quality with sufficient compute budget
• Significant domain shift requiring updating all weights

Skill page

Source