cyberine
UsageRun recipes

Anthropic chat prompt caching (kv0qh)

The chat planner puts cachecontrol breakpoints on the trailing system block and the last tool definition of every request whose provider routes to an Anthropic model (the same policy as the...

The chat planner puts cache_control breakpoints on the trailing system block and the last tool definition of every request whose provider routes to an Anthropic model (the same policy as the fleet's loop workers, src/core/cachePolicy.ts). Annotations are added only on the request body; the stored history stays plain. Anthropic's own OpenAI-compatible endpoint (https://api.anthropic.com/v1/) ignores cache_control and reports no cached tokens, so chat there is billed as a full miss on every call. Route Anthropic chat through OpenRouter or a LiteLLM gateway instead.

CYBERINE_CHAT_PROVIDER=openrouter CYBERINE_CHAT_MODEL=anthropic/claude-haiku-4.5 cyberine chat
cyberine doctor    # "Chat prompt cache:" shows annotated / automatic / unsupported / unknown for the configured planner

Measured 2026-10-09 through plannerTurn on OpenRouter anthropic/claude-haiku-4.5 with a 15k-token prefix: call 1 cached 0 (the write), calls 2 and 3 cached 14,973 tokens.

Known Limitations and Deferred Work. The prefix must reach the model's minimum cacheable length (about 1,024 to 4,096 tokens depending on the model); a shorter prompt is processed uncached with no error. The 5-minute TTL applies unless the account is promoted; a pause longer than that starts cold. The chat callers do not yet keep a CachePolicySession across turns (PlannerTurnOptions.cacheSession), so the latch counter and the read-file breakpoint choices restart each turn. cyberine doctor reports the planner's configured lane only; it does not probe the provider.