Token & cost optimization
2 min read · Hugging Face Blog
Quantization-Aware Healing Recovers 4-Bit LLMs Beyond Full-Precision Performance
Quantization-Aware Healing (QAH) recovers compressed 4-bit models by distilling directly from the original full-scale teacher rather than intermediate checkpoints. Applied to a GPT-OSS 120B model compressed to 60B in MXFP4, it outperforms its own 16-bit bfloat16 source on 7 out of 9 benchmarks.
Why it matters. Engineers can now deploy half-sized, 4-bit models that deliver higher accuracy and lower latency than their 16-bit checkpoints.