How LLM Statistical Watermarking Works and Why Light Paraphrasing Fails
An interactive breakdown illustrates how models like Claude and Gemini embed statistical watermarks by subtly biasing candidate token selection using secret keys. Detection relies on k-gram window matches, meaning only full semantic rewrites successfully erase the mark.
Impact: Medium
Why it matters
Understand how model-level sampling watermarks behave during generation and how window-based detection resists superficial edits.
TL;DR
- 01Watermarks are embedded during model sampling without adding metadata or changing characters.
- 02Detectors require the provider's private secret key to verify green-token statistical bias.
- 03Only full semantic re-composition breaks contiguous n-gram windows enough to reset detection.
Key facts
- KGW Window Survival
- 0.5% after full rewrite
- Detection Threshold
- ~1,500 words for subtle shifts
- Key Ownership
- Model provider required
Token Sampling and Secret Keys
During sentence generation, language models evaluate a shortlist of plausible next tokens. Watermarking algorithms use a secret key based on preceding tokens to color candidate words red or green, tilting selection probability toward green tokens.
Detection Mechanics and Window Thresholds
Detection does not use stylistic heuristics; it recalculates token colors with the key and measures green token frequencies:
- Short texts & code: Offer insufficient statistical slack for watermark insertion.
- Long text (>1,500 words): Provides high statistical confidence even with minimal probability shifts.
- Light paraphrase: Fails to remove marks because contiguous n-gram windows survive.
- Full semantic rewrite: Erases marks by breaking >99.5% of original token window sequences.
✓ When to use
- When evaluating whether light text editing erases LLM generation artifacts
- When designing pipelines to audit or verify AI-generated documentation
✕ When NOT to use
- When analyzing source code or deterministic data structures where word choice has zero variance
What to do today
- Audit automated content generation workflows to ensure full semantic recomposition when clean output is required.
Sources