domains / families / research / nanogpt

nanogpt

separate

Educational GPT implementation in ~300 lines. Reproduces GPT-2 (124M) on OpenWebText. Clean, hackable code for learning transformers. By Andrej Karpathy. Perfect for understanding GPT architecture from scratch. Train on Shakespeare (CPU) or OpenWebText (multi-GPU).

When to use

• Learning how GPT works
• Experimenting with transformer variants
• Teaching/education purposes
• Quick prototyping
• Limited compute (can run on CPU)
• **~300 lines**: Entire model in `model.py`
• **~300 lines**: Training loop in `train.py`
• **Hackable**: Easy to modify

When not to use

• Production use
• Large-scale distributed training
• High-level framework needs

Skill page

Source