Your AI Feature Does Not Cost What the Pricing Page Says

The question that arrives with the second invoice is always the same: why is this so much higher than the first one? The answer is usually not that usage grew. It is that the first invoice covered a pilot running on curated inputs with a short prompt, and the second one covered a live system running on real inputs with a prompt that has been growing every week since launch.

Nobody asks about the cost curve at the start because the pricing page makes it look like arithmetic. Take the token price, multiply by expected volume, done. That calculation is correct and it describes maybe half of what you will pay.

Cost is driven by input, and input grows on its own

Most people size an AI feature by thinking about the answer. The answer is short. The thing you send in order to get it is not, and it is usually charged at a comparable rate.

A support-reply feature starts as a prompt with a few instructions and a customer message. Then someone adds three example replies to fix tone. Then the retrieval step starts attaching four knowledge-base articles instead of one, because recall was poor. Then the conversation history gets included so the model stops repeating itself. Six weeks after launch, the input for a single reply is many times the size it was at launch, and nothing about the feature looks different from the outside.

This growth is not a mistake. Each addition was a reasonable fix for a real quality problem. But quality improvements in these systems are almost always paid for in input length, and if nobody is tracking input size per request as a metric, the cost increase arrives as a surprise rather than as a decision. Our guide to what AI features actually cost to run breaks the components down further.

The invisible multipliers

Below is what tends to sit between the naive estimate and the invoice. None of it is exotic.

Cost driver	Why it is missed	What it does
Retries on malformed output	Assumed rare in the pilot	Each retry is a full-price call, and structured-output failures cluster on hard inputs
Retrieval calls	Counted as free	Embedding every query and every document update carries its own charge
Context growth	Not measured per request	Input size drifts upward with every quality fix
Evaluation runs	Treated as development	A test suite run on every change is production traffic against the provider
Chained calls	Modelled as one step	A classify-then-extract-then-summarise flow is three billed calls, not one
Human review	Counted as headcount, not feature cost	Often the largest line, and it is the one that scales worst
Failed and abandoned work	Not counted at all	A user who rejects the output still paid for generation

The last three are where the real money is. A feature at a low unit price that requires a person to check every output has a true cost dominated by that person, and it will not get cheaper as volume rises. It will get more expensive in a straight line, which is the opposite of the software economics the business is expecting.

Measure cost per resolved outcome, not cost per call

Cost per call is a number you can compute and it tells you almost nothing. The useful denominator is the completed unit of work: a ticket actually resolved, an invoice actually posted, a document actually filed without a human touching it.

Consider two designs for the same task. One uses a small model, costs very little per call, and gets it right often enough that a human corrects a meaningful share of outputs. The other uses a larger model, costs several times more per call, and requires review far less often. Per call, the first is obviously cheaper. Per resolved outcome, the second frequently wins once you price the reviewer's time, and it wins by more as volume grows.

This is why per-token comparisons between providers are close to useless as a procurement input. The rate difference between reasonable options is small next to the difference in how often each one produces something a human has to fix.

The same logic decides the hosting question. Self-hosting looks cheaper at high, steady volume and considerably worse at low or spiky volume, because you are paying for idle hardware rather than idle capacity. The comparison of hosted APIs against self-hosted models sets out where the crossover sits and what else changes with it.

Controls to put in before launch, not after

Five things, all cheap to build at the start and awkward to retrofit.

1. Log input and output token counts per request alongside the request itself. Without this you cannot attribute a cost rise to anything, and the first cost investigation will be archaeology.
2. Set a hard cap on input size, with truncation rules you have chosen deliberately. Uncapped retrieval and uncapped conversation history are the two most common ways a bill compounds.
3. Cache aggressively on identical or near-identical inputs. In document and support workloads, repeated inputs are far more common than teams expect.
4. Route by difficulty. Send the easy majority to a smaller model and escalate only what fails a confidence check. This requires an evaluation set to establish where the line sits, which is another reason testing an AI integration pays for itself early.
5. Set a per-tenant or per-workflow budget alarm that fires on a daily threshold rather than a monthly one. Monthly alarms tell you about the problem after you have paid for it.

What to do next

Take whatever estimate you are working from and rebuild it around one number: cost per completed outcome, including the human minutes that outcome consumes. Then run the same calculation at ten times your current volume, with the input size doubled, and see whether the feature still makes sense. If it only works at pilot scale with a short prompt, you have a demo rather than a product.

If the answer is uncomfortable, the fix is usually narrower scope rather than a cheaper model. A feature that handles the sixty percent of cases that are genuinely routine, and routes the rest to a person immediately, tends to have far better economics than one that attempts everything and needs checking everywhere. Our AI integration services work starts from that sizing question rather than ending at it.