domains / families / data / deepspeed

deepspeed

separate

Expert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention

When to use

• Working with deepspeed
• Asking about deepspeed features or APIs
• Implementing deepspeed solutions
• Debugging deepspeed code
• Learning deepspeed best practices

Skill page

Source