Results for “Prompt caching”

Why it matters

NVIDIA Research has introduced HORIZON, an agentic framework for RTL design that achieves 100% on hardware benchmarks. By hosting problems as git worktrees and using persistent sessions, it achieves a 91% prompt caching rate.

Why it matters

Amazon Bedrock launched openai.gpt-6-astra across two distinct endpoints: bedrock-runtime and bedrock-mantle. Selecting an endpoint forces an architectural trade-off between explicit prompt caching and centralized invocation audit logging.

Why it matters

Developers often encounter unexpectedly high API bills because they fail to configure and utilize prompt caching correctly. Understanding how token state, system prompts, and history affect cache hits can drastically cut costs.

Why it matters

Google announced Gemini 4 Argon, a frontier reasoning model featuring an unprecedented 1 million output token limit. Designed for long-horizon engineering and autonomous refactoring, it introduces 95% prompt caching discounts and sets new state-of-the-art benchmarks in SWE tasks.

Why it matters

AI agent token consumption is projected to grow twenty-four-fold by twenty-thirty. Developers must master context optimization strategies like prompt caching to manage application budgets. Stay cost-efficient.

Why it matters

Anthropic's massive valuation highlights the intense monetization pressure building on foundation model providers. For developers, this accelerates the rollout of hyper-efficient billing features like prompt caching.

Why it matters

A systematic analysis of developer agents reveals that Claude Code injects around 33,000 tokens of system prompts, tool schemas, and reminders before reading your query, compared to OpenCode's minimal 7,000 tokens. This overhead escalates with system instructions, custom AGENTS.md files, and active Model Context Protocol servers, heavily impacting prompt-caching efficiency.

Why it matters

An 8-week analysis of 1,138 Claude Code prompts revealed that prompt-caching context re-reads account for 96% of total token volume. Average agentic sessions process 2.9 million tokens across 12 model calls per user prompt.

Why it matters

Cursor has shared a prompt for improving the token efficiency of AI agent harnesses. One round of prompt trimming, tool offloading, and cache layout changes cut one production team's overall token cost by about 7% with no loss in quality. Offloading non-core tools cut tool-description tokens by 60%, and making Model Context Protocol schemas discoverable via grep or jq cut total session tokens by 46.9% in sessions that used them.

Why it matters

Developers using native OpenAI Codex CLI via Amazon Bedrock Mantle on GPT-5.6 Sol report high costs driven by missing explicit prompt cache controls. Telemetry shows cache writes accounting for nearly 85% of total model spend due to missing request options.

Why it matters

DeepSeek has updated its API pricing structure, introducing differential rates for peak and off-peak demand periods alongside revised calculations for prompt cache hits. AI developers should adjust scheduled execution windows to optimize API expenditures.

Why it matters

Anthropic introduced Claude Fable 5.1, offering deeper reasoning, agentic coding capabilities, and a 1M token context window. Prompt cache reads are reduced to a quarter of the prior cost alongside several breaking changes to thinking blocks and tool calls.