What Is Prompt Caching? Cut Claude/OpenAI API Costs in Production with Go

When developers who connect Claude or OpenAI to their Go services ask our team, "Our API bill jumped this month. Should we switch to a cheaper model?", our most common answer is "Not yet. First look at what you send again on every request." Most systems resend the same long system prompt, the same tool definitions and the same reference documents on every call, and pay full price for them every single time.

Both providers already have a fix for this. It is called Prompt Caching. But from what we have seen, plenty of systems turn it on and save nothing at all, because the cache never actually hits.

This article answers one question: how do you shape requests from Go so the cache hits every time it should, and how do you prove from the numbers that it is really cutting your costs?

How Prompt Caching works

There is exactly one principle: prefix match. The provider keeps the processed result of the beginning of your prompt. If the next request starts with exactly the same content, character for character, that part does not have to be processed again and is billed at a much lower rate.

"Character for character" is the most important phrase in this article. If anything earlier in the prompt differs by even one character, such as a timestamp, a user name or the order of fields in JSON, everything after that point is a miss.

Claude builds the prefix in a fixed order: tools, then system, then messages (Claude docs). Change a tool definition even slightly and the cache for everything after it is gone.

How Claude and OpenAI differ

• Turning it on: Claude needs you to add cache_control yourself, at the request level or on individual blocks, with up to 4 breakpoints. OpenAI caches automatically with no code changes, and also offers explicit breakpoints.

• Minimum size: On Claude it depends on the model, for example 1,024 tokens for Sonnet 5 and 512 tokens for Opus 5.5. On OpenAI it is 1,024 tokens for GPT-5.6 and later.

• Cache write cost: Claude charges 1.25x the input price for a 5-minute cache or 2x for a 1-hour cache. OpenAI charges nothing for models before GPT-5.6 and 1.25x from GPT-5.6 onward.

• Cache read cost: Claude charges 0.1x the input price on most models. OpenAI discounts cached input by up to about 90%, depending on the model.

• Cache lifetime: On Claude it is 5 minutes, refreshed for free every time the cache is used, with an optional 1-hour tier. On OpenAI the system manages it for you.

• Rate limits: On Claude, cache hits are not deducted from your rate limit. On OpenAI, cached tokens still count toward tokens-per-minute. If you keep hitting quotas, see EP.164 Rate Limiting AI Requests.

Sources: Claude docs and OpenAI docs (as of September 2026; multipliers and models change when new models ship. For an overview of the latest Claude models, see Claude Opus 5 for developers).

In short, OpenAI is easier because you get caching without doing anything, while Claude gives you finer control over exactly what gets cached. That control pays off when your prompt has several parts that change at different rates.

Enabling Claude Prompt Caching from Go

This example uses net/http and plain structs so you can see exactly what JSON goes over the wire. To compare with calling OpenAI through its SDK, see EP.144 Connecting to the OpenAI API with the Go SDK.

package llm

import (
	"bytes"
	"encoding/json"
	"fmt"
	"net/http"
	"os"
)

type CacheControl struct {
	Type string `json:"type"`          // "ephemeral"
	TTL  string `json:"ttl,omitempty"` // "" = 5 minutes, "1h" = 1 hour
}

type TextBlock struct {
	Type         string        `json:"type"`
	Text         string        `json:"text"`
	CacheControl *CacheControl `json:"cache_control,omitempty"`
}

type Message struct {
	Role    string `json:"role"`
	Content string `json:"content"`
}

type Request struct {
	Model     string      `json:"model"`
	MaxTokens int         `json:"max_tokens"`
	System    []TextBlock `json:"system"`
	Messages  []Message   `json:"messages"`
}

type Usage struct {
	InputTokens              int `json:"input_tokens"`
	CacheCreationInputTokens int `json:"cache_creation_input_tokens"`
	CacheReadInputTokens     int `json:"cache_read_input_tokens"`
	OutputTokens             int `json:"output_tokens"`
}

type Response struct {
	Usage Usage `json:"usage"`
}

// systemPrompt must be static: no timestamps, user names or anything that changes per request
func Ask(systemPrompt, question string) (*Response, error) {
	body := Request{
		Model:     "claude-sonnet-5",
		MaxTokens: 1024,
		System: []TextBlock{{
			Type:         "text",
			Text:         systemPrompt,
			CacheControl: &CacheControl{Type: "ephemeral"}, // breakpoint at the end of the static part
		}},
		Messages: []Message{{Role: "user", Content: question}}, // the changing part comes after the breakpoint
	}

	b, err := json.Marshal(body)
	if err != nil {
		return nil, err
	}

	req, err := http.NewRequest("POST", "https://api.anthropic.com/v1/messages", bytes.NewReader(b))
	if err != nil {
		return nil, err
	}
	req.Header.Set("x-api-key", os.Getenv("ANTHROPIC_API_KEY"))
	req.Header.Set("anthropic-version", "2023-06-01")
	req.Header.Set("content-type", "application/json")

	res, err := http.DefaultClient.Do(req)
	if err != nil {
		return nil, err
	}
	defer res.Body.Close()
	if res.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("anthropic: status %d", res.StatusCode)
	}

	var out Response
	return &out, json.NewDecoder(res.Body).Decode(&out)
}

The heart of this code is a single line: CacheControl must sit on the last block that is identical on every request, never on a block that changes each time.

Reading the numbers

The usage object in the response has three values to watch.

• cache_creation_input_tokens is the number of tokens just written to the cache in this request.

• cache_read_input_tokens is the number of tokens read from the cache in this request.

• input_tokens is only the tokens after the last cache breakpoint, not your total input.

Total input is the sum of all three. If cache_creation_input_tokens and cache_read_input_tokens are both 0, nothing was cached. The usual cause is a prompt shorter than the model's minimum; the API silently skips caching without returning an error (Claude docs).

On OpenAI, read cached_tokens from usage.prompt_tokens_details in Chat Completions or usage.input_tokens_details in the Responses API. GPT-5.6 and later also report cache_write_tokens (OpenAI docs).

Is it worth it? Working it out from real prices

Take Claude Sonnet 5 prices per million tokens from the docs as of September 2026: $2 for regular input, $2.50 for a 5-minute cache write and $0.20 for a cache read.

Assume a 10,000-token system prompt and 100 requests arriving often enough that the cache never expires.

• Without cache: 100 x 10,000 x $2 / 1,000,000 = about $2.00

• With cache: one write at 10,000 x $2.50 / 1,000,000 = $0.025, plus 99 reads at 99 x 10,000 x $0.20 / 1,000,000 = $0.198, for a total of about $0.22

This covers only the system prompt. It excludes user questions and output, which cost the same either way, and it assumes every request hits. A real system will save less, in proportion to its hit rate.

Break-even comes very quickly: one write plus one read costs 1.25 + 0.1 = 1.35x, while two uncached requests cost 2x. A single hit already pays for the write. The function below calculates this from real usage data and plugs straight into the cost-logging setup from EP.149 Token Management.

// Price per million tokens (USD). Pull it from the pricing page for the model you use; do not hardcode it forever.
type Price struct {
	Input, CacheWrite5m, CacheRead float64
}

func CostUSD(u Usage, p Price) (withCache, withoutCache float64) {
	const m = 1_000_000.0
	withCache = float64(u.InputTokens)*p.Input/m +
		float64(u.CacheCreationInputTokens)*p.CacheWrite5m/m +
		float64(u.CacheReadInputTokens)*p.CacheRead/m

	total := u.InputTokens + u.CacheCreationInputTokens + u.CacheReadInputTokens
	withoutCache = float64(total) * p.Input / m
	return
}

This function covers input only and assumes the 5-minute cache. If you use the 1-hour cache, price cache_creation.ephemeral_1h_input_tokens separately.

You get speed as well as savings. OpenAI's own tests found that short 1,024-token prompts got about 7% faster, while prompts of 150,000 tokens or more saw roughly 67% faster time-to-first-token (OpenAI Cookbook). The longer the prompt, the bigger the difference.

Can I just put cache_control on every request?

You can, and Claude has an automatic caching mode: set cache_control once at the request level and the API moves the breakpoint to the last block for you. It is a great fit for multi-turn chats whose history keeps growing.

But if the last block changes every time, for example because you append the current time or per-request context to the end of the prompt, the API writes a new cache entry on every request and never reads one. You pay a 25% write premium on every request and get nothing back (Claude docs). In that case, use an explicit block-level breakpoint placed at the end of the static part instead.

The easy rule to remember: what never changes goes first, what changes goes last. A safe order is tools, core instructions, reference documents, chat history, and finally the latest question.

A trap Go developers need to know about

The Claude docs warn that some languages, Go among them, may randomize key order when converting to JSON, which breaks cache matching.

It is worth being precise here. Go's encoding/json always sorts map keys in json.Marshal, so that path is safe. The real risks are elsewhere.

• Building a prompt by ranging over a map. Map iteration order in Go is randomized, so the resulting text comes out in a different order each time. Use a slice with a fixed order instead.

• Using another JSON library that does not guarantee key order.

• tool_use input you store and send back, if you convert it through a map and reassemble it without sorting the keys.

The safest approach is to use structs, as in the example above, because struct field order is always fixed.

Does caching make the AI give the same answer, or use stale data?

This is worth asking before you turn it on. The answer is no. Prompt caching has no effect on output generation; the response is identical to what you would get without caching (Claude docs). What the cache stores is the processed input, not the answer.

Caching does not keep your data fresh either, though. If the reference documents in your prompt are out of date, answers will still be based on the old content. Systems that need current data still need their own way to update that content, and must accept that every update causes one miss for that section and everything after it.

Teams also often ask about security. Claude caches are isolated between organizations, and on the Claude API also between workspaces. OpenAI caches are isolated per organization. Users in different organizations never share a cache.

Summary: turning the cache on is not enough, you have to design for hits

The straight answer:

Do it now if your system has a system prompt, tool definitions or reference documents longer than your model's minimum, and requests arrive often enough that the cache does not expire. A single hit already pays for the write.

No need to rush if your prompts are shorter than the minimum, or every request is completely different. Caching will barely help there.

If you start today, start with one thing: add cache_read_input_tokens or cached_tokens to the logs you already have and watch for a day how often you really hit. That number will tell you what to move to the front of your prompt and where you are still leaking.