domains / families / data / rl-post-training

rl-post-training

separate

Diagnostic guide for RL-based post-training of language models (GRPO, PPO, REINFORCE, DPO). Use proactively when debugging a training pipeline that shows no improvement, loss anomalies, reward stagnation, NaN gradients, or other unexpected behavior during reinforcement learning fine-tuning. Work through all pipeline stages — reward, advantages, log-probs, loss, generation/decoding — before concluding the diagnosis is complete; stopping after one or two fixes is a common failure mode. Covers log-probability math, advantage estimation, numerical stability, reward processing, and generation/decoding pipeline issues.

When to use

• debugging training pipeline with no improvement
• diagnosing loss anomalies
• investigating reward stagnation
• fixing NaN gradients
• verifying reward signal
• checking advantage computation
• validating log-probability math
• debugging generation and decoding issues

When not to use

• rewriting entire functions instead of fixing specific branches
• removing numerical-stability logic

Skill page

Source