domains / families / data / nemo-mbridge-perf-moe-long-context

nemo-mbridge-perf-moe-long-context

separate

Long-context MoE training guidance for Megatron Bridge. Covers CP sizing, selective recompute, dispatcher choices, and practical patterns from DSV3, Qwen3, and Qwen3-Next long-context experiments.

When to use

• Training MoE at long sequence lengths
• Investigating long-context MoE OOM
• Investigating degraded throughput
• CP sizing for long sequences
• Selective recompute long context

Skill page

Source