domains / families / data / nemo-mbridge-perf-parallelism-strategies

nemo-mbridge-perf-parallelism-strategies

separate

Operational guide for choosing and combining parallelism strategies in Megatron Bridge, including sizing rules, hardware topology mapping, and combined parallelism configuration.

When to use

• Choosing or sizing TP/DP/PP/CP/EP degrees
• Tracing an OOM or regression to a parallelism config change
• How to parallelize
• Tensor parallel
• Pipeline parallel
• Parallelism config
• Which parallelism for X GPUs

Skill page

Source