Prompt caching in 2025: how do you cut cost with Anthropic & OpenAI caches?

Q: Prompt caching in 2025: how do you cut cost with Anthropic & OpenAI caches?

**Target Audience:** Engineers who already know what prompt caching is and now have to make it pay on a real bill, on Anthropic, OpenAI or both. **Difficulty:** Medium **Verified with:** Node 22.22.0 and Python 3.12.3 (the cost simulator below). Provider behaviour was checked in October 2026 against the Anthropic docs, OpenAI announcements, and the Azure and third-party documentation cited inline where OpenAI's own guide could not be read directly. > **How to read this doc:** Every idea is explained in plain language first, then tagged with `📌 Interview term:` so you also learn the words an interviewer expects. All program output shown was produced by actually running the example; the **Verified** section quotes it in full. The title says 2025; the facts below are as of October 2026, and several changed during 2025 and 2026 (longer lifetimes, deeper discounts, explicit modes). The mechanism underneath, an exact-prefix reuse of the model's attention state, is explained in [What is pr

The problem

Question presented to candidate: "We run the same agent on Claude and on an OpenAI model. Each request carries about 10,000 tokens of tools, instructions and reference docs, and caching is barely showing up on either bill. How do the two providers' prompt caches actually work, and what would you change to cut the cost?"

What a strong answer should cover:

The shared mechanism and the shared rule: both reuse an exact prompt prefix, so the prompt must be laid out stable-first (tools, system, documents, history) with per-request data last
Anthropic specifics: explicit cache_control breakpoints (up to 4) or one automatic top-level cache_control, the tools-system-messages order, a 5-minute lifetime refreshed on each hit or a 1-hour option, writes billed at 1.25x or 2x and reads at a fraction of the input price, model-specific minimum lengths
OpenAI specifics: automatic caching from 1,024 tokens in 128-token steps with no code change, routing by a hash of the prompt start plus an optional prompt_cache_key, in-memory versus extended (24-hour) retention, and the newer explicit-breakpoint mode on recent models
Economics: a write surcharge means caching is a bet on reuse inside the lifetime; picking the lifetime from the traffic pattern (busy, sparse, bursty) and computing the break-even
Measurement: reading cache_read_input_tokens / cache_creation_input_tokens or cached_tokens per request, tracking hit rate per prompt version, and catching the deploy that silently breaks the prefix

Clarifying questions expected:

What does the traffic look like per prompt prefix: steady, bursty, or a few requests an hour?
Which parts of the prompt change per tenant, per user and per request?
Which exact models are in use (minimum cacheable length, discount and retention options are per model)?

Code / implementation expected: Optional — request shapes for both providers with the cache controls, plus the cost arithmetic for the chosen lifetime.

prompt-caching
cost

Try it yourself

Sketch your approach before you read the solution. That is what the interview tests.

Stuck? Get a hint.

A short nudge toward the approach. It will not give the answer away.

Solution

Target Audience: Engineers who already know what prompt caching is and now have to make it pay on a real bill, on Anthropic, OpenAI or both. Difficulty: Medium Verified with: Node 22.22.0 and Python 3.12.3 (the cost simulator below). Provider behaviour was checked in October 2026 against the Anthropic docs, OpenAI announcements, and the Azure and third-party documentation cited inline where OpenAI's own guide could not be read directly.

How to read this doc: Every idea is explained in plain language first, then tagged with 📌 Interview term: so you also learn the words an interviewer expects. All program output shown was produced by actually running the example; the Verified section quotes it in full.

The title says 2025; the facts below are as of October 2026, and several changed during 2025 and 2026 (longer lifetimes, deeper discounts, explicit modes). The mec

The solution is hidden so you can try first

18 min read · 12 sections

Read the code

Cache policy cost simulator: write fee and lifetime vs traffic pattern

Open in playground

// Cost of one shared prompt prefix under four cache policies, replayed against three
// traffic patterns on a fake clock (minutes). Multipliers are relative to the base input
// price, in hundredths (100 = 1x), so all arithmetic stays in integers.
const PREFIX = 10000, SUFFIX = 200;     // tokens: shared prefix, per-request tail
const PRICE = 3.0;                       // $ per million input tokens (illustrative)

const POLICIES = [
  { name: "no cache", ttl: 0,    write: 100, read: 100, steps128: false },
  { name: "write 1.25x, 5m", ttl: 5,    write: 125, read: 10,  steps128: false },
  { name: "write 2x, 1h", ttl: 60,   write: 200, read: 10,  steps128: false },
  { name: "free write, 5m", ttl: 5,    write: 100, read: 10,  steps128: true },
  { name: "free write, 24h", ttl: 1440, write: 100, read: 10,  steps128: true },
];

// Caches that bill in steps count hits from 1,024 tokens in 128-token steps.
function steppedCachedTokens(prefix) {
  if (prefix < 1024) return 0;
  return 1024 + Math.floor((prefix - 1024) / 128) * 128;
}

function simulate(policy, times) {
  let expires = -1, hits = 0, units = 0;
  for (const t of times) {
    const hit = policy.ttl > 0 && t < expires;
    if (hit) {
      hits++;
      const c = policy.steps128 ? steppedCachedTokens(PREFIX) : PREFIX;
      units += c * policy.read + (PREFIX - c) * 100;
    } else {
      units += PREFIX * policy.write;
    }
    units += SUFFIX * 100;
    if (policy.ttl > 0) expires = t + policy.ttl;      // a write or a hit refreshes the TTL
  }
  return { hits, dollars: units / 100 * PRICE / 1e6 };
}

const range = (start, stop, step) => { const r = []; for (let t = start; t < stop; t += step) r.push(t); return r; };
const PATTERNS = {
  "busy: 1 req/min for 1h": range(0, 60, 1),
  "sparse: every 8 min for 2h": range(0, 120, 8),
  "bursts: 5 in a row every 30 min": range(0, 120, 30).flatMap(s => [s, s + 1, s + 2, s + 3, s + 4]),
};

console.log(`prefix ${PREFIX} tokens, suffix ${SUFFIX}, base $${PRICE.toFixed(2)}/M; 128-step caches count ${steppedCachedTokens(PREFIX)} of ${PREFIX} prefix tokens`);
for (const [label, times] of Object.entries(PATTERNS)) {
  console.log(`\n${label} (${times.length} requests)`);
  const base = simulate(POLICIES[0], times).dollars;
  for (const p of POLICIES) {
    const r = simulate(p, times);
    const delta = (100 * (r.dollars / base - 1)).toFixed(1);
    console.log(`  ${p.name.padEnd(18)} hits ${String(r.hits).padStart(2)}/${times.length}  $${r.dollars.toFixed(4)}  ${(delta.startsWith("-") ? "" : "+") + delta}% vs no cache`);
  }
}

// Break-even reads per write: extra write cost divided by saving per read.
for (const p of POLICIES.slice(1, 3)) {
  const reads = Math.ceil((p.write - 100) / (100 - p.read));
  console.log(`\n${p.name}: pays off once a write is followed by ${reads} read(s) inside the TTL`);
}

Discussion

Other approaches, follow-ups and how it went in real interviews.

Sign in to join the discussion.

No comments yet. Share how you would answer it.

The solution is hidden so you can try first

Keep your momentum going.