Boundary-Aware Distillation Cuts Large Language Model False Refusals to Four Percent
Multiverse Computing introduced boundary-aware self-distillation to prevent LLMs from rejecting benign queries that share vocabulary with dangerous topics. By incorporating benign boundary training data, over-refusal on permissible queries dropped from 32.9% to 4.2%.
Impact: Medium
Why it matters
Standard safety alignment often acts like a blunt instrument, refusing benign requests simply because they touch sensitive topics. Boundary-aware tuning preserves model utility on legitimate domain queries while maintaining safety guardrails.
TL;DR
- 01Safety tuning that optimizes solely for harm refusal can increase false refusal on benign queries up to 74%.
- 02Escalating retry strategies reduce training sample dropout from 19.88% to 0.20%, preserving the hardest prompts.
- 03Incorporating benign boundary data reduces false refusals on boundary test pairs from 32.94% to 4.16% while retaining 87.72% harmful refusal.
Key facts
- Over-refusal drop (boundary data)
- 32.94% to 4.16%
- Harmful refusal rate retained
- 87.72% (from 91.88%)
- Training drop rate reduction
- 19.88% down to 0.20% via escalated retry
- Mean unsafe response rate (HarmBench, StrongREJECT, WildJailbreak)
- 26.26% down to 0.14%
The Over-Refusal Trap in Safety Tuning
Standard safety pipelines steer models to refuse prompts across entire domains. On Qwen3-8B, basic self-generated safety data reduced the mean unsafe-response rate across three harmfulness benchmarks (HarmBench, StrongREJECT, and WildJailbreak) from 26.26% down to 0.14%. However, the same checkpoint caused benign query over-refusal on XSTest to skyrocket from 2.00% to 74.00%.
Boundary-Aware Pair Architecture
To build sharp refusal boundaries without collateral damage, Multiverse Computing introduced three data pipeline fixes:
1. Coverage Escalation: Single-shot generation dropped 19.88% (8,009) of hard prompts. Escalating retry strategies through progressively stronger steering reduced residual dropouts to 0.20% (79 prompts), preserving 40,293 training samples. 2. Target-Generated Benign Traces: Replacing external compliance data with target model self-generated responses dropped XSTest over-refusal from 15.20% to 5.20%. 3. Benign Boundary Compensation: Adding benign boundary data dropped over-refusal on the comply-worthy side of 1,539 held-out evaluation pairs from 32.94% to 4.16%, while maintaining harmful refusal at 87.72% (down slightly from 91.88%).
Metric Balance
Evaluating safety models solely on refusal rates hides critical UX degradation. Testing both sides of the boundary confirms that deliberate data composition preserves model utility without sacrificing safety.
✓ When to use
- Deploying models in domains like politics, law, or healthcare where benign prompts use sensitive vocabulary.
- Calibrating refusal behavior when general safety guardrails trigger unacceptable false refusal rates.