Prompt Caching

How does Prompt Caching work: what is a "cache prefix" and why does it dramatically reduce costs?

Claude API billing is per token — each API call's input token count determines input cost; output token count determines output cost.

In many applications, each API call's "first half" (System Prompt, background documents, role setup) is identical; only the "second half" (user's question) changes each time. Without Prompt Caching, every API call sends the complete System Prompt to Anthropic's servers for recomputation — even if identical.

Prompt Caching works by: on the first API call, Anthropic's servers compute and store the System Prompt's "intermediate computation state" (KV Cache); subsequent calls just read this cache without recomputing. Cache read cost is 10% of original computation cost — meaning those tokens' cost drops 90% on cache hits.

Cache TTL is 5 minutes, reset on each hit. As long as your API call frequency exceeds once every 5 minutes, the cache stays effective.

Example cost calculation: 3,000-token System Prompt + 100-token user question. Original: (3,000 + 100) × $0.003/1K = $0.0093/call. With caching on cache hit: 3,000 × $0.003 × 10% + 100 × $0.003 = $0.0009 + $0.0003 = $0.0012/call — 87% savings.

How do you correctly implement Prompt Caching in the API? What are common implementation mistakes?

Correct implementation:

response = client.messages.create(
    model="claude-sonnet-4-5",
    system=[
        {
            "type": "text",
            "text": "Your fixed System Prompt content (must exceed 1,024 tokens)",
            "cache_control": {"type": "ephemeral"}  # This enables caching
        }
    ],
    messages=[{"role": "user", "content": user_question}]
)

# Check cache status
print(response.usage.cache_creation_input_tokens)  # First call: has value
print(response.usage.cache_read_input_tokens)       # Subsequent: has value

Common implementation mistakes:

Mistake 1: System Prompt too short (under 1,024 tokens) — Anthropic's minimum cache requirement is 1,024 tokens; adding cache_control has no benefit.

Mistake 2: Cache marker added to content that changes every time. cache_control is effective for all content before the marked text, and cache only hits when content is exactly identical. If you mark content containing dynamic user information, the cache will never hit.

Mistake 3: Not verifying cache is actually hitting. Check cache_read_input_tokens in the response usage field; if always 0, cache isn't hitting — check configuration.

What application scenarios are best suited for Prompt Caching? How to calculate expected cost savings?

Best scenarios for Prompt Caching:

Multi-turn conversation applications — same System Prompt each round, only user messages and assistant replies accumulate. One of the highest-benefit caching scenarios since the System Prompt only needs computing once per conversation.

Knowledge base Q&A (RAG) — long knowledge base document in System Prompt; different users' questions use the same document as context. Knowledge base documents may be 50,000-100,000 tokens; caching savings are very significant.

Batch processing — analyzing large volumes of documents (100 contracts, 1,000 emails) with identical analysis instructions. Instructions serve as cache prefix; each document as user input.

Cost savings estimation formula:

Expected monthly savings = (System Prompt tokens × call count × cache hit rate × 0.9 × input rate)

Simplified estimate: 2,000-token System Prompt, 10,000 monthly API calls, 90% cache hit rate (call frequency higher than once per 5 min), Sonnet 4.5 ($0.003/1K input): Monthly savings ≈ 2,000 × 10,000 × 0.9 × 0.9 × $0.003 / 1,000 = $48.6.

This shows: for high-frequency API applications, Prompt Caching savings are substantial — far exceeding setup costs (a few minutes of code changes).

How to combine Prompt Caching with other cost optimization strategies (model downgrading, output length control) for best effect?

Prompt Caching is one of several main Claude API cost optimization tools, working better in combination:

Combination 1: Prompt Caching + Tiered routing

Use Haiku for classification routing (determining request complexity), then select Sonnet or Opus by complexity. Enable Prompt Caching separately for each model. This combination typically reduces costs 60-75%: Haiku's low cost handles routing, Prompt Caching reduces System Prompt costs for each model, tiered routing reduces high-cost model usage.

Combination 2: Prompt Caching + Conversation history management

For long conversations, control history within reasonable bounds (compress old history with summaries) while enabling caching for System Prompt. Caching handles fixed prefix costs; history management controls dynamic part growth.

Combination 3: Prompt Caching + Batch API

Anthropic's Batch API lets you submit many requests at once at 50% of real-time API cost (processing takes longer; suitable for non-real-time tasks). Combined with Prompt Caching, batch processing cost savings can reach 80-90%.

Practical impact on your API costs: if your monthly Claude API costs exceed $100, evaluate optimizations in this order: first check if Prompt Caching is enabled (5-minute setup, fastest savings); then analyze request distribution to evaluate tiered routing feasibility; finally consider Batch API for non-real-time tasks.