Optimizing Context Windows with OpenAI Server-Side Compaction
OpenAI introduced Server-Side Compaction to reduce context size in long-running interactions while preserving critical conversation state. This stateless feature allows developers to maintain reasoning quality while lowering latency and token costs.
Impact: High
Why it matters
You can drastically reduce your API costs and long-tail latency in long agentic conversations by letting the server handle context pruning automatically.
TL;DR
- 01Compaction reduces active context sizes in long conversations while preserving key reasoning state.
- 02Enabled by passing context_management with compact_threshold inside response creation calls.
- 03In stateless chaining, dropping items before the latest compaction item reduces payload size and long-tail latency.
- 04It is fully ZDR-friendly when combined with the store=false configuration.
Key facts
- Trigger threshold config
- compact_threshold
- API Endpoint
- POST /responses or responses.create
- Stateless Endpoint
- /responses/compact
Configuring Server-Side Compaction
To enable server-side compaction, configure the context_management parameter within your response creation request. You set a compact_threshold representing the token limit at which compaction triggers.
Integration Patterns
You can manage your conversation loops using one of two patterns: 1. Stateless Input-Array Chaining: Append the assistant's output, including the encrypted compaction items, to your next request's input array. To minimize request size and latency, you can safely drop all items that occurred before the most recent compaction item. 2. Previous Response ID Chaining: Pass only the new user message each turn alongside the previous_response_id. In this mode, do not manually prune items, as the server handles the chain automatically.
Both patterns remain Zero Data Retention (ZDR) friendly if you specify store: false in your request payload.
Try it in 2 minutes
next_response = client.responses.create(
store=False,
model="gpt-4o",
context_management=[{
"type": "compaction",
"compact_threshold": 200000
}],
input=[*compacted_output, {"role": "user", "content": "Continue coding tasks..."}]
)python
✓ When to use
- When building long-running chat applications or coding assistants that accumulate large histories.
- When trying to balance token costs and latency in multi-turn interactions.
- When you need to preserve reasoning context without maintaining full transcript payloads on the client.
✕ When NOT to use
- For short, single-turn requests or simple Q&A tasks where context limits are never approached.
- If you require perfect, word-for-word recall of earlier messages rather than synthesized, compacted state.
What to do today
- Update your OpenAI API client configurations to include the context_management property in long-running agent workflows.
- Set an appropriate compact_threshold based on your budget and expected maximum context size.
- Implement stateless input pruning by dropping items older than the returned encrypted compaction item to optimize latency.
Sources