Stable FP4 Training via Transposition-Invariant Block Quantization
Mehdi Rahimifar, Amin Darabi, Mehran Taghian Jazi, Xing Huang, Yao Wang, Zhijun Tu, Yufei Cui, Yunke Peng, Hongliang Li
Identifies tensor-transposition-induced scale inconsistency as a key cause of FP4 training instability and proposes a 2D block quantization scheme with transposition-invariant scaling. Achieves stable end-to-end FP4 training within 1.3% of BF16, validated on LLMs up to 7B parameters and a 30B MoE model.
- FP4
- Low-precision training
- LLMs