AWS, OpenAI, and Google all quietly discount AI requests that reuse the same document or instructions, and most teams never ask whether their tools pass that savings through.
A team that keeps asking an AI tool follow-up questions about the same contract, policy manual, or product catalog is normally paying to have that entire document reread from scratch every time, even though nothing in it changed between questions. Amazon detailed this week how Amazon Bedrock now lets a model skip that rereading and bills the recognized portion at a steep discount, up to 90 percent less on the tokens it has already seen, according to AWS's own writeup. The same discount, it turns out, already exists inside the AI tools built on OpenAI's and Google's models too, whether or not anyone told you about it.
By the end of this, you should be able to look at any AI tool your team pays for, ask one specific question about how it handles repeated content, and find out whether you are quietly overpaying for the same information every time someone asks it a new question.
What the concept actually is. The idea is called prompt caching, or context caching in Google's case. Every time you send a question to an AI model, you are also sending everything it needs in order to understand that question: the document you want it to read, the instructions that tell it how to behave, the list of tools it is allowed to use. If none of that surrounding material changes between questions, the model can remember it instead of processing it fresh, and several AI vendors now charge less for the parts it already recognizes. That is the entire mechanic. No new model, no new capability, just a different way of billing for the same request.
The walkthrough. AWS frames the problem with a plain example: send a 10,000-token contract to a model along with 50 different questions about it, and without caching you are billing 500,000 input tokens, the whole contract, 50 separate times, as if it were new content every single time (AWS's blog post). Bedrock's pricing page lists the rates for one of its Claude models this way: standard input tokens cost $6.00 per million, writing content into the cache for the first time costs 25 percent more at $7.50 per million, and reading content back out of the cache on every question after that costs 90 percent less at $0.60 per million.
Run those rates against AWS's own 50-question example and the difference is concrete. Reread the contract 50 times at the standard rate and that piece of the bill comes to about $3.00. Write it into the cache once, then read it from cache the other 49 times, and the same 50 questions cost roughly $0.37, close to an 88 percent reduction on that portion of the bill. Nothing about the questions or the answers changes. Only the billing does.
This is not a Bedrock-only trick. OpenAI has discounted cached input tokens automatically since October 2024: send GPT-4o the same content twice within a few minutes and the repeated portion drops from $2.50 to $1.25 per million tokens, a straight 50 percent cut, with nothing to configure (OpenAI). Google's Gemini API does something similar under the name context caching. On Gemini 2.5 and newer models, implicit caching is on by default, and if a prompt reuses the same block of content, roughly the first 2,048 to 4,096 tokens depending on the model, Google applies the savings automatically and reports the cached token count back in the response (Google). Bedrock's own version of this covers Claude and Amazon's Nova models, since Bedrock is where those models are actually hosted and billed.
Anyone who has built a custom assistant with a long set of instructions, what tone to use, what topics to avoid, what format to answer in, is already generating exactly the kind of repeated content caching is built for. AWS's post walks through caching a persona definition that runs several thousand tokens long, the kind of system prompt a team might write for a customer service assistant or an internal research tool, and reusing it across every question instead of paying to reprocess it on every single message. The same principle covers the list of tools an AI agent is allowed to call, which also tends to stay fixed for the length of a conversation. None of this requires touching model weights or retraining anything. It is closer to how a person would naturally work: read the manual once, then answer questions about it, rather than rereading the manual before answering each one.
For a business leader, the practical upshot is that AI cost is not simply a function of how many questions your team asks. It is a function of how repetitive the surrounding context is and whether the tool you are using is built to reuse it. A tool that reprocesses your entire knowledge base on every question will cost meaningfully more to run at scale than one that caches it, even if both are built on the exact same underlying model. That is a real point of comparison the next time you are evaluating AI vendors: not just which model they use, but whether they use it efficiently.
The pitfall. None of this happens automatically inside most of the AI tools your team actually clicks around in day to day. Prompt caching is a feature of the underlying model API, not of every product built on top of it. If your team uses an AI-powered helpdesk tool, a contract review assistant, or an internal chatbot that some vendor built on GPT-4o, Claude, or Gemini, caching only saves your team money if that vendor implemented it and structured requests to take advantage of it. Whether that savings reaches your invoice or just widens the vendor's margin is not something you can tell from the outside. Caches are also short lived. Amazon's default cache window is five minutes and tops out at an hour for select models, OpenAI clears its cache after five to ten minutes of inactivity and always within an hour, and Google's caching only helps when similar requests land close together in time. Starting a brand new conversation instead of continuing an existing one, or letting a chat sit idle too long, quietly turns the discount back off.
Try this today. Ask your AI vendor, in writing, whether prompt caching or context caching is enabled on the model behind your product, and whether the savings actually show up in what you are charged. If your team uses a raw AI chat tool directly, built on Claude, GPT-4o, or Gemini, keep follow-up questions about the same document inside one ongoing conversation instead of opening a new chat each time, since a fresh conversation forces the model to reread everything from zero. If your team regularly uploads the same policy document, product manual, or dataset and asks it a series of questions over a session, that is exactly the workload this feature was built for, and it is worth flagging to whoever manages that tool.
The interesting part is not the discount itself. It is that AI pricing, billed by the token regardless of whether that token is new, has quietly assumed for a while now that most of what you send a model is information it has never seen. It usually is not. I would be curious how many line items on your AI vendor invoices are actually charging you full price to reread something you showed it five minutes ago.