Why Falling Token Prices Didn't Lower Your Agent Bills

Table of Contents

• The Paradox in One Sentence
• What Actually Got Cheaper (and What Didn't)
• Jevons' Paradox Comes for Your Inference Budget
• The Six Multipliers That Ate Your Savings
• Reasoning Tokens You Never See
• Retries and the Silent Retry Tax
• Context Bloat and Re-Sending the Same Tokens
• Tool Calls That Compound
• Multi-Agent Fan-Out
• The Shift From Cheap Models to Frontier Models
• How to Actually Read Your Agent Bill
• What This Means for GaaS Buyers and Sellers
• Insights Most People Overlook
• References

The Paradox in One Sentence

A token costs a tenth of what it did, and your monthly agent bill is higher than ever. Both things are true at the same time, and once you see why, you stop expecting the next price cut to rescue your margins.

I've watched this confuse smart finance teams over and over. The CFO sees a headline that GPT-class inference dropped another 60%, pencils in a budget cut, and three months later the line item has grown. Nobody is lying to them. The unit got cheaper while the number of units exploded, and in agentic systems the number of units is not a constant you control by hand. It's an emergent property of how the agent decides to work.

That's the whole game in this corner of the GaaS cluster: the economics of agents are governed less by sticker price per token than by how many tokens a task actually burns, and that count has been climbing fast.

What Actually Got Cheaper (and What Didn't)

Per-token prices have genuinely fallen, and not by a little. Across the major providers, the cost to process a million input tokens on a flagship-tier model has dropped by something like an order of magnitude since 2023, and small/distilled models are cheaper still. Stanford's 2024 AI Index report documented inference cost for a GPT-3.5-equivalent capability falling by more than 280-fold over roughly 18 months. That number is real and it's enormous.

Here's what didn't get cheaper: the amount of work an autonomous agent decides to do to finish a task. When you bought a chatbot completion in 2023, you paid for one request and one response, call it 2,000 tokens round trip. When you deploy an agent to "resolve this support ticket end to end" in 2026, that single outcome might touch a planning step, four tool calls, three retrieval passes, two self-critique loops, and a verification pass, tens of model calls and hundreds of thousands of tokens before it returns an answer.

So the price per token fell maybe 10x. The tokens per task rose 50x. You do the arithmetic. The bill goes up, and the per-token discount is real the entire time. This is the core distinction the rest of the cluster keeps circling: cost-per-completed-task, not cost-per-token, is the unit that actually shows up on your invoice.

Jevons' Paradox Comes for Your Inference Budget

Economists have a name for this. In 1865, William Stanley Jevons noticed that as steam engines got more efficient at burning coal, England burned more coal, not less, because efficiency made coal-powered everything cheaper and therefore more widespread. Cheaper unit cost expanded the use cases faster than efficiency reduced per-use consumption. Microsoft's CEO invoked exactly this framing about AI compute in early 2025, and he was right to: it's the cleanest mental model we have for what's happening to agent bills.

Falling token prices didn't make people run the same workloads for less. They made entirely new workloads economically viable. A multi-step research agent that would have cost $40 to run in 2023 cost $4 in 2025, and at $4, teams started running it 30 times a day across the whole org instead of twice a week in a skunkworks project. The per-run cost dropped 10x and the spend went up 7x because demand was sitting there, latent, waiting for the price to cross a threshold.

This is the part budget models almost always miss. They treat token price as exogenous and volume as fixed. In agentic systems, price drives volume. Cut the price and you don't bank the savings, you induce more consumption, because the agent is now cheap enough to point at problems you'd never have automated before.

The Six Multipliers That Ate Your Savings

If Jevons explains the macro story, these six mechanisms explain the line items. Each one independently inflates tokens-per-task, and they stack multiplicatively.

Reasoning Tokens You Never See

The single biggest change since 2024 is the rise of reasoning models that "think" before answering. These models generate long internal chains of thought, sometimes thousands of tokens, that you are billed for as output, even though most providers don't surface the full reasoning trace to you. A query that once produced 200 visible output tokens can now bill you for 4,000 tokens of hidden deliberation plus the 200 you see.

Reasoning models genuinely improve agent reliability, which is why teams adopt them. But the trade is explicit in the pricing: you pay per-token for thinking. The provider docs are upfront about it, Anthropic and OpenAI both bill reasoning/thinking tokens as billable output. If your agent quietly switched to a reasoning model for hard steps, your token count per task can 5-10x while the per-token price is unchanged or lower. The headline says tokens got cheaper; your agent just started buying a lot more of them per decision.

Retries and the Silent Retry Tax

Agents fail and retry. A tool call times out, a JSON parse fails, a downstream API returns a 429, a verification step rejects the first answer. Every retry re-runs some or all of the prior context. In a poorly instrumented system, a task that "succeeded" might have made fifty model calls, forty-nine of which were dead ends you still paid for. This retry tax is invisible on a per-token price sheet and brutally visible on the bill, it's significant enough that it gets its own node in this beat.

Context Bloat and Re-Sending the Same Tokens

Here's the mechanic that surprises people most: in a multi-turn agent loop, the entire conversation history is re-sent as input on every single step. Step 1 sends 5,000 tokens. Step 2 sends step 1 plus the new stuff: 9,000. By step 12, you might be re-sending 80,000 input tokens per call, and you pay for all of them every time. A 12-step agent doesn't cost 12x a single call, because of accumulating context, it can cost 40-60x. Prompt caching helps enormously here when it's configured, but most teams leave that lever half-pulled, which is exactly why caching and memory get treated elsewhere in the cluster as primary margin levers rather than nice-to-haves.

Tool Calls That Compound

Every tool call is a round trip: the model emits a call, you execute it, you feed the result back in, the model reasons over it. Each leg is billable, and tool results (a 20-page API response, a full webpage, a database dump) often dwarf the prompt itself in token volume. An agent that uses ten tools to complete a task isn't paying for ten cheap calls, it's paying for ten growing context windows, each carrying the accumulated weight of everything before it.

Multi-Agent Fan-Out

The architectural fashion of 2025-2026 is orchestrator-plus-subagents: a lead agent spawns specialized workers that each run their own full loop. This is great for quality and catastrophic for naive budgeting. Anthropic reported that its multi-agent research system used about 15x the tokens of a plain chat interaction. Fifteen times. A token price cut of 50% against a 15x architecture leaves you paying roughly 7.5x more than the single-agent baseline. Fan-out economics deserve their own model, and they get one elsewhere in this beat, but the headline is simple: spawning sub-agents is the fastest way to outrun any price cut.

The Shift From Cheap Models to Frontier Models

Finally, blended rate. Teams don't run one model, they run a mix, and the mix has drifted upward in capability because agents need reliability and frontier models deliver it. Even as each individual model got cheaper, the weighted average price per token your agent pays can rise if it's now routing the hard 30% of steps to the most expensive frontier tier. The cheapest model on the menu got cheaper; your agent stopped ordering it for the steps that matter.

How to Actually Read Your Agent Bill

If you take one operational thing from this piece, make it this: stop tracking cost-per-token and start tracking cost-per-completed-task, then decompose it. Your bill is the product of four things, and you need visibility into all four:

Tasks attempted × steps per task × tokens per step × blended price per token.

Token price, the only number that fell, is one of four multiplicands, and it's the one you don't control. The other three are architecture decisions you own. A useful audit looks like this:

• Instrument tokens-per-successful-task as a first-class metric. Not tokens total, not tokens per call, tokens per outcome you actually billed the customer for. This is the number that reveals whether your agent is getting more efficient or quietly bloating.
• Separate productive tokens from retry/dead-end tokens. If 40% of your spend is failed attempts, that's a reliability problem wearing a cost-problem costume, and the fix is engineering, not a cheaper model.
• Audit context re-sending. Check whether prompt caching is actually configured and hitting. A cache miss rate you didn't know about can be a third of your bill.
• Watch the blended rate, not the cheapest tier. Track what fraction of steps route to frontier models and whether that fraction is creeping up.

When you slice it this way, the paradox dissolves. You can literally watch the per-token price falling in column four while columns one through three climb fast enough to swamp it. That's not a billing anomaly. That's your architecture telling you where the money goes.

What This Means for GaaS Buyers and Sellers

For buyers of Agentic AI-as-a-Service, the lesson is to stop negotiating on token price and start negotiating on outcome price. A vendor bragging that they pass through cheaper model costs is telling you they have no idea what their tokens-per-task curve looks like, and that curve, not the spot price of inference, determines what you pay. Ask any GaaS vendor for their cost-per-completed-task trend over the last year. If it's flat or rising while token prices crater, their architecture is eating the savings, and eventually that shows up in your invoice.

For sellers, this is the entire margin story. As McKinsey has noted in its work on the economic potential of generative AI, value capture depends on the gap between what a workflow is worth and what it costs to run, and that cost is governed by tokens-per-task, not token price. If you priced your service on the assumption that falling token costs would lift your margins automatically, you built your model on the one variable that doesn't behave the way you hoped. The vendors who win the GaaS margin game in 2026 aren't the ones who got the best inference discount. They're the ones who drove tokens-per-task down through caching, smart model routing, retry suppression, and disciplined context management, because that's the multiplicand they actually control.

The deflation in token prices is real and it's ongoing. It's just not a refund. It's an invitation to do more, and agentic systems accepted that invitation enthusiastically.

Insights Most People Overlook

• Cheaper tokens make agent bills less predictable, not more. Lower prices lower the barrier to running an agent on marginal, low-confidence tasks, exactly the tasks that retry, loop, and fan out unpredictably. The deflation doesn't just raise spend; it widens the variance of spend, which is why finance teams who got comfortable with flat SaaS bills hate this category.

• Provider price cuts can quietly raise your blended rate. When a provider cuts the price of its frontier tier, it becomes economical to route more steps there instead of to the cheap model. A price cut on the expensive option can pull your whole mix upmarket, increasing average cost per token even as every individual model got cheaper. The cut changes behavior before it changes the bill.

• Reasoning models inverted the old optimization. For years the advice was "shorten your prompts." With reasoning models, the expensive tokens are output the model generates internally to think, and you can't shorten those by editing your prompt. The lever moved from prompt engineering to deciding when reasoning is worth paying for at all, which is a completely different (and harder) discipline.

• The retry tax is a margin problem disguised as a reliability metric. Teams track success rate as a quality KPI and cost as a finance KPI and never connect them. But a 70% first-attempt success rate means roughly 30% of your token spend is funding failures. Reliability is unit economics in this category, they're the same line on the P&L viewed from two angles.

• "Pass-through pricing" is a confession, not a feature. A GaaS vendor who promises to pass model-cost savings straight to you is admitting their value-add doesn't reduce tokens-per-task. The vendors actually building durable margin keep the efficiency gains they engineer, because those gains came from their orchestration work, not from the provider's price sheet.

References

• Stanford HAI, 2024 AI Index Report
• Anthropic Engineering, How We Built Our Multi-Agent Research System
• McKinsey, The Economic Potential of Generative AI: The Next Productivity Frontier