Why Per-Token AI API Pricing Is Broken

DEV Community

AI API Pricing Is Broken: Per-Token Billing Charges You for the Wrong Thing

Originally published at API Cost Lab. AI API Pricing Is Broken: Per-Token Billing Charges You for the Wrong Thing Last updated: November 2026 Per-token pricing is the wrong unit for AI APIs. It charges you for effort, not for results — for every token a model burns on the way to an answer, including the tokens you never see, the retries that failed, and the words a rambling answer didn't need. The fix is not a cheaper token. It is a different unit: price per finished task, not per token spent getting there. That sounds like a small accounting complaint. It isn't. The unit you pay by decides what you optimise for, what you fear, and what your provider is rewarded for. Right now, all three point the wrong way. You pay for the journey, not the arrival Think of a taxi ride. You pay for arriving, with the meter as a rough proxy for distance. Now imagine a taxi that charged per turn of the wheels — extra when the driver took a wrong turn, circled the block, or sat thinking at a junction with the engine running. That is how an AI API bill works today. Every call is metered in tokens, and a token is not a unit of value. It is a chunk of text, roughly part of a word. You are billed for the prompt you send, for the answer you receive, and for everything the model generates in between — including work that produces nothing you can use. A call that fails halfway still costs you the tokens it consumed. An agent loop that retries a step five times bills you five times for one job. A model that answers in 400 words costs roughly twice as much as one that answers the same question correctly in 200. The meter runs on effort, and effort is the one thing you cannot see or control in advance. This was tolerable when an API call was one prompt and one answer. It is not tolerable now that a single user request can set off a chain of dozens of calls, each feeding the next, with the total unknown until the chain stops. Our bill-diagnosis checklist walks through the usual ways a bill runs away, and nearly all of them are the same story: a loop nobody capped, a prompt nobody trimmed, a retry nobody counted. Per-token pricing turns every one of those slips into money, silently, at machine speed. Exhibit one: the tokens you never see The clearest proof that the token is the wrong unit is the reasoning token. When a reasoning model works through a hard problem, it generates an internal draft — its working-out — before producing the answer you actually read. OpenAI's own documentation says those reasoning tokens are not visible through the API, but they still occupy context space and are billed as output tokens, at the output price. Read that again: the most expensive tokens on the bill are the ones you are not allowed to read. The same documentation notes that a model may generate anywhere from a few hundred to tens of thousands of reasoning tokens for one problem. You asked for a short answer and paid for a hidden essay you will never see. Reasoning effort settings make the absurdity explicit. Turn the effort dial up and the same question, with the same correct answer, costs more — because the model was allowed to think longer. The answer didn't change. The value didn't change. Only the amount of invisible work changed, and the meter charges for invisible work at the highest rate on the price sheet. A unit that bills you more for the same answer, depending on a dial you set blind, is not measuring value. It is measuring its own process. Exhibit two: the same answer at five different prices If the token were a real unit of value, the same answer would cost the same. It doesn't — not even close. The price of an identical response swings wildly depending on plumbing choices that have nothing to do with what you received. Send a prompt whose beginning matches a cached prefix and, on Anthropic's published terms, those tokens are read back at a tenth of the standard input price. Write a new cache entry and the same tokens cost a quarter more than standard ...

Powered by Readybot.io