AI API Token Pricing Explained: How Input, Output and Cached Tokens Affect Your Bill
AI API pricing can look simple until a real application starts generating a bill. Providers often quote a price per million tokens, but the total can depend on several different buckets: uncached input, cached input, output, reasoning-related usage, images, audio, tools or batch processing. Understanding those pieces is essential before comparing models or estimating what an AI feature will cost at scale.
Table of Contents
01 What It Is
A token is a unit a model uses to process text and other encoded information. It is not exactly a word. A short word may be one token, while a longer word, punctuation or part of a word can be split differently. That means “1,000 words” and “1,000 tokens” are not interchangeable.
API providers typically meter usage separately for input tokens and output tokens. Input includes the instructions, conversation history, documents and other context sent to the model. Output is what the model generates. Output is frequently priced higher because generation requires sequential computation.
Some providers also identify cached input tokens. When a repeated prefix or large block of context can be reused from a cache, those tokens may be billed at a lower rate than ordinary input. The precise pricing and eligibility rules vary by provider and model.
02 How It Works
The basic cost equation is straightforward: multiply the number of tokens in each pricing category by that category’s rate, then add the results. The complexity comes from understanding what falls into each bucket.
Imagine an application that sends a long system instruction, several pages of reference material and a short user question. All of that can count as input. If the model writes a 700-token answer, those generated tokens count as output. If the repeated instruction and reference prefix qualify for caching, part of the input may be charged at a cached rate.
Conversation history matters too. A chat application that resends the entire conversation on every turn can accumulate a large input bill as the thread grows. Retrieval systems can have a similar problem if they stuff too many documents into every request. Reducing irrelevant context can therefore improve both cost and answer quality.
03 Why It Matters
The cheapest model on a price table is not automatically the cheapest model for a completed task. A weaker model might require more retries, longer prompts, extra verification or human correction. A more expensive model may complete the task correctly in one pass. The useful measure is often cost per successful outcome, not cost per million tokens.
This is becoming more important as companies look harder at AI spending. Earnyx has covered how business AI budgets are starting to face more scrutiny. API pricing is where abstract model economics turn into an operating expense that can rise with every user, conversation and automated workflow.
Token pricing also shapes product design. Developers may route simple tasks to cheaper models, summarize long histories, use retrieval instead of sending entire databases, cache repeated prompts or process non-urgent jobs in batches.
04 What It Means for You
If you are building with an AI API, start by measuring real usage rather than estimating from prompt length alone. Providers expose usage metadata that can show input tokens, output tokens and, in some APIs, cached-token details. Log those values by feature so you know which workflows are actually expensive.
Then separate fixed context from variable context. A long instruction block that appears in thousands of requests may be a good candidate for prompt caching. Large documents should be retrieved selectively rather than inserted in full if only a few passages are relevant. Output length controls can also matter because generated tokens may carry the highest text rate.
If you are buying AI software rather than building it, token pricing still matters indirectly. Subscription limits, credit systems and usage caps often reflect the vendor’s underlying inference costs. Earnyx has an existing AI pricing topic in the database for consumer-facing credit systems; API token pricing is the developer-side economics underneath those products.
05 Numbers + Context
Prices change often, so an evergreen comparison should avoid treating today’s model rate as permanent. The better framework is to compare four numbers for each workload: average input tokens, average output tokens, percentage of input that receives a cached rate, and task success rate.
For example, suppose two models have very different published rates. If Model A costs half as much per token but succeeds 70% of the time while Model B succeeds 95% of the time, the raw price difference overstates the real savings. Add retries and human review and the cheaper model may no longer be cheaper.
Modern API usage records can break out cached input from total input, making cost attribution more precise. OpenAI’s usage documentation, for example, distinguishes input tokens and cached input tokens in reporting. Always use the current provider pricing page before committing a production budget because rates and model lineups can change.
06 Earnyx Takeaway
AI API bills are not simply “prompt cost plus answer cost.” They are the result of how much context you send, how much the model generates, whether repeated context can be cached, which model receives the request and how often the workflow has to retry.
The most useful optimization is not blindly shortening every prompt. Measure cost per successful task, remove context that does not help, cache genuinely reusable material, route workloads intelligently and keep current pricing outside your application logic where it can be updated. Token efficiency matters—but outcome efficiency matters more.
