domains / families / science / nemo-mbridge-perf-memory-tuning

nemo-mbridge-perf-memory-tuning

separate

Techniques for reducing peak GPU memory in Megatron Bridge — expandable segments, PEFT + SP input re-gather, parallelism resizing, activation recompute, CPU offloading constraints, and common OOM fixes.

When to use

• GPU OOM errors
• reducing peak memory
• reducing LoRA or PEFT activation memory with sequence parallelism
• tracing an OOM regression to a specific commit or config change
• memory fragmentation
• expandable_segments
• reduce GPU memory
• LoRA memory

When not to use

• increasing TP as a memory fix
• increasing PP at the cost of DP
• CPU offloading when PP > 1

Skill page

Source