domains / families / data / mhc-algorithm

mhc-algorithm

separate

Implement mHC (Manifold-Constrained Hyper-Connections) for stabilizing deep network training. Use when implementing residual connection improvements with doubly stochastic matrices via Sinkhorn-Knopp algorithm. Based on DeepSeek's 2025 paper (arXiv:2512.24880).

When to use

• | Scenario | Approach | |----------|----------| | Standard residual connection | No mHC needed | | Deep networks (>12 layers) with stability issues | Use mHC with num_streams=4 | | GPT/Transformer training | Wrap both attention and MLP wit…

When not to use

• Standard residual connections

Skill page

Source