Prompt Caching

• Two Approaches
• Setup Workflow
• Key Concepts
• Minimum Token Thresholds
• Why Isn’t My Cache Working?
• Debug Workflow
• Break-Even Analysis
• Preventing Cache Fragmentation

Two Approaches

Simplified (Claude models only): A single cachePoint marker; Bedrock checks ~20 preceding blocks automatically. First request shows cacheWriteInputTokens > 0; subsequent identical requests show cacheReadInputTokens > 0.

Explicit (all supported models): Place multiple cachePoint markers at specific positions. Supports mixed TTL (1h + 5min) for different content sections.

Setup Workflow

1. Choose Strategy

Ask the developer which approach fits. Simplified is recommended for Claude-only workloads. Explicit is required for Nova models or mixed-TTL scenarios.

2. Fetch Implementation Guidance

Before giving implementation advice, fetch the latest from the aws-samples repo:

• Fetch https://raw.githubusercontent.com/aws-samples/amazon-bedrock-samples/main/introduction-to-bedrock/prompt-caching/README.md
• Key directories: converse_api/ (recommended), invoke_model_api/ (provider-specific)

3. Configure TTL

When mixing TTLs, longer durations MUST precede shorter ones.

4. Validate

Confirm cache write on first request and cache read on second.

Key Concepts

The cachePoint is a standalone content block placed after the content to cache: {"cachePoint": {"type": "default"}}. For 1-hour TTL, add "ttl": "1h".

Cache metrics in the Converse API usage object:

• cacheWriteInputTokens > 0: Cache populated (first request or expired)
• cacheReadInputTokens > 0: Cache hit (subsequent requests within TTL)
• Both zero: Below threshold or unsupported model

For InvokeModel (Anthropic format): cache_creation_input_tokens and cache_read_input_tokens.

Good candidates: System prompts, few-shot examples, reference docs, tool definitions, long code files. Poor candidates: Per-request user messages, dynamic context, content below the token threshold.

Minimum Token Thresholds

Content before a cache point must meet the model’s minimum. Below threshold = silently ignored.

Why Isn’t My Cache Working?

Caching fails silently. Checklist:

1. Model not supported? Silently ignored for unsupported models.
2. Below minimum threshold? Cache point ignored if content is too short.
3. Content not identical? Cache keys use exact byte-for-byte prefix match. Invalidators: timestamps in system prompts, whitespace differences, reordered JSON keys, session tokens before the cache point.
4. TTL expired? Default is 5 minutes. After expiry, next request is a cache write.
5. Cache point misplaced? Must be a separate content block placed after the content to cache.

Debug Workflow

Run 6 automated diagnostic tests when cache issues are reported:

Tests: (1) Model support, (2) Token threshold, (3) Cache write/read cycle, (4) Prefix sensitivity, (5) TTL behavior, (6) Break-even analysis.

If tests fail: Focus on the matching section above. Prefix sensitivity failures indicate cache fragmentation (see below). Break-even failures mean caching is not cost-effective at the developer’s request volume.

After diagnosis: Recommend simplified vs explicit caching for their model, 5-min vs 1-hour TTL for their request pattern, and whether caching is cost-effective.

Break-Even Analysis

Cache writes cost 25% more than standard input tokens. Cache reads cost 90% less.

You need at least 2 requests within the TTL window to break even. For single-use content, do NOT enable caching.

Preventing Cache Fragmentation

Cache fragmentation = “static” content varies between requests. Fixes:

• Move timestamps and session IDs AFTER the cache point
• Separate static content from dynamic user context
• Use sorted JSON keys, consistent whitespace, fixed-format strings