domains / families / data / ray-train

ray-train

separate

Distributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when training massive models across multiple machines or running distributed hyperparameter sweeps.

When to use

• Training across multiple machines (multi-node)
• Need hyperparameter tuning at scale
• Want fault tolerance (auto-restart failed workers)
• Elastic scaling (add/remove nodes during training)
• Unified framework (same code for PyTorch/TF/HF)
• **Multi-node orchestration**: Easiest multi-node setup
• **Ray Tune integration**: Best-in-class hyperparameter tuning
• **Fault tolerance**: Automatic recovery from failures

Skill page

Source