The Blended-Rate Illusion: Why Averaging Model Costs Quietly Wrecks Agent Pricing

Table of Contents

• What "blended rate" actually means in a multi-model agent
• Why the average is the wrong number
• The routing-mix drift problem
• Variance is the cost, not the mean
• How blended rates get weaponized in pricing decks
• A worked example: the support agent that looked 70% margin
• What to measure instead of a blended rate
• Insights Most People Overlook
• References

What "blended rate" actually means in a multi-model agent

Almost no serious agent runs on one model anymore. A coding agent might draft with a mid-tier model, escalate to a frontier model for the gnarly refactor, and fire a tiny classifier model a hundred times to decide which file to open. A support agent might triage on something cheap and fast, then call an expensive reasoning model only when the customer's problem clears a confidence threshold. This is good engineering. Routing to the cheapest model that can do the job is the single biggest lever on agent gross margin, and the teams that do it well genuinely deserve their margins.

The trouble starts at the spreadsheet. To set a per-task price, someone has to turn this messy routing behavior into one number: dollars per completed task. So they take a sample of tasks, sum the model spend, divide by task count, and call it the blended rate. Maybe it's $0.34 a task. They mark it up to $1.20, declare a 70%-ish gross margin, and move on.

That blended rate is an average over a distribution. And the distribution is the whole story.

Why the average is the wrong number

Here is the uncomfortable property of agent workloads: model cost per task is not normally distributed. It's heavily right-skewed, often closer to a power law. The median task is cheap. The mean is dragged upward by a long tail of tasks that loop, retry, escalate, and fan out into sub-agents. When a distribution is skewed like that, the average describes a task that almost never happens, it sits in the sparse valley between "most tasks are cheap" and "a few tasks are brutally expensive."

You can see why this matters with a trivial example. Imagine 100 tasks. Ninety of them cost two cents. Ten of them, the ones that triggered three retries on a frontier model with a big context window, cost a dollar each. Your blended rate is $0.118. But no single task cost twelve cents. Your pricing is anchored to a phantom. And critically, your margin is now hostage to the count of those ten tasks, which you do not control. The customer controls it, by sending you hard work.

This is the same trap cloud-cost teams fell into a decade ago, and the lesson translates cleanly: as a16z argued in its widely-cited piece on the cost of cloud and the trillion-dollar paradox, variable infrastructure that scales with usage quietly becomes the dominant line item, and averaging it away is how margins erode without anyone noticing in time.

The routing-mix drift problem

The blended rate has a second, sneakier failure mode. It assumes the mix of models is stable. It isn't.

Your router sends easy work to cheap models and hard work to expensive ones. That means the model mix is a function of the difficulty of incoming work, which drifts constantly. A new customer cohort with messier data shifts the mix toward the frontier model. A product launch that surfaces edge cases shifts it. A prompt-injection wave, a seasonal spike in complex tickets, a single enterprise client whose use case happens to be your worst case, all of these silently re-weight the blend.

So the $0.34 you measured in a clean quarter is not a constant. It's a snapshot of one mix at one moment. The next quarter, the same agent doing the "same" task type might cost $0.51 because 8% more of the volume escalated to the expensive tier. Nothing in your code changed. Your blended rate just told you a stale story. This is closely related to the token-volatility problem, except here the swing comes not from provider price changes but from your own routing logic reacting to demand. The cost moves even when prices are frozen.

The deeper point: in a multi-model agent, your COGS is set by an interaction between your customers' behavior and your router's policy. A single blended number erases both variables.

Variance is the cost, not the mean

If you remember one thing, make it this. In a usage-priced agent business, your exposure isn't the average task cost, it's the variance.

A 70% gross margin at the mean can be a loss at the 95th percentile. If you've priced flat per-task and a meaningful slice of tasks costs 6x the average, those tasks are sold below cost. You're cross-subsidizing your hardest, often most valuable customers with your easy ones. That's survivable when the tail is thin and stable. It becomes existential when a big customer's workload is the tail, because now you've signed a contract priced on a blended rate that their specific usage will never resemble.

Frontier-model pricing makes this sharper, because the expensive tier is genuinely expensive and the gap between tiers is large. Provider rate cards, see Anthropic's published model pricing for a current sense of the spread between fast/cheap and frontier reasoning models, show order-of-magnitude differences per million tokens. When your router can swing a task from the cheap tier to the frontier tier, a single routing decision can multiply that task's cost by 10x or more before you count retries and reasoning tokens. The blend smooths all of that into a flat line that hides exactly where you're bleeding.

The right mental model is an insurer's, not an accountant's. You are not pricing the average claim. You are pricing the distribution of claims, and you survive or die on the tail.

How blended rates get weaponized in pricing decks

Blended rates don't just mislead by accident. They get used because they make the deck look good.

Watch for these moves. A vendor reports "cost per task" without reporting its distribution, no median, no p90, no p99. A board slide shows a smooth, flat per-task COGS line that has no business being smooth given how lumpy agent work is. A pricing model that promises to "just pass through model costs" while quoting a single rate, which, as the margin trap of cost pass-through shows, only works if the pass-through tracks the customer's actual tail, not your historical average. Or the classic: a margin number computed on a cohort of tasks that conveniently predates the customer whose hard workload is about to blow up the blend.

None of this requires bad intent. A founder genuinely believes the 70% number because the spreadsheet said 70%. The spreadsheet said 70% because it averaged. The honest version of the slide shows a cost distribution and names the percentile where margin goes negative.

A worked example: the support agent that looked 70% margin

Make it concrete. A customer-support agent priced at $1.00 per resolved ticket. The team measures blended model cost at $0.30, 70% gross margin, ship it.

Pull the distribution apart and it looks like this. Sixty percent of tickets are simple FAQs resolved on a cheap model in one pass: $0.04 each. Thirty percent need a reasoning model and a tool call or two: $0.35 each. Ten percent are nasty, multi-turn, several escalations, a frontier model chewing through a long conversation with retries: $2.20 each. Weighted, that's $0.024 + $0.105 + $0.22 = $0.349. Round to $0.30-ish after some caching wins. The blended rate checks out.

Now look at what you actually sold. On the cheap 60%, you make $0.96 a ticket, fantastic. On the middle 30%, $0.65, fine. On the nasty 10%, you make negative $1.20 per ticket. You are paying customers to send you hard tickets. As long as the mix holds, the easy tickets bankroll the hard ones and the blend works.

Then a fintech client signs. Their tickets are disputes and account-security issues, almost all of them land in your nasty 10% bucket. Now 70% of their volume costs $2.20 against your $1.00 price. That single logo, priced on the blended rate, runs at roughly negative 120% gross margin. Nothing broke. The router worked perfectly. The blended rate simply never described that customer, and you didn't notice until the invoice from your model providers arrived. This is also why forecasting under per-task pricing is so hard, the unit cost is conditional on a customer mix you learn about only after you've signed the contract.

What to measure instead of a blended rate

You don't have to abandon per-task pricing. You have to stop pricing against the mean and start pricing against the distribution. Practically:

Report the full cost distribution per task type. Median, p90, p99, and the max. The mean is the least useful of the five. If your p99 is 15x your median, you have a tail-risk pricing problem no average will reveal.

Track the model-mix ratio over time as a first-class metric. What fraction of tasks touch the frontier tier this week versus last? That ratio is your margin's leading indicator. Alert on it the way you'd alert on error rate. McKinsey's work on the economics of generative AI repeatedly lands on the same point, that capturing real value from gen AI depends on disciplined cost tracking at the workflow level, not blended top-line estimates.

Segment cost by customer, not just by task type. The blended rate's worst lie is at the customer level, because customers are not random samples of your task distribution, each one is a biased sample shaped by their use case. A per-customer cost-to-serve number catches the fintech-client problem before the contract, not after.

Price the tail explicitly. Tiered pricing, complexity surcharges, per-outcome pricing that scales with task difficulty, or hard autonomy caps that stop a runaway task before it eats your margin, any of these beats a flat rate sold against an average. Some vendors are quietly capping autonomy specifically to protect margin, which is really just tail-risk management wearing a product hat.

Run the worst-case spend scenario, not the expected one. Model what happens if your mix shifts 20% toward the expensive tier and your retry rate doubles at the same time. If that scenario turns your best customer into a loss, your blended rate was never a real margin, it was a bet that the tail would stay polite.

Insights Most People Overlook

The blended rate gets more dangerous as your router gets smarter. Counterintuitive, but true. A better router widens the gap between your cheap and expensive paths, it sends only the genuinely hard work to the frontier model instead of wasting it on easy tasks. That improves your average and your median. It also makes your tail more concentrated and more expensive per task, because the expensive bucket is now pure hard work with nothing cheap mixed in to soften it. Optimizing the mean can fatten the tail. Your blended rate improves while your tail risk gets worse.

Your blended rate is a competitor's acquisition strategy. If you price flat against an average, a competitor can cherry-pick your easy-task customers with aggressive pricing they can afford because they aren't carrying your tail, and leave you holding the hard, unprofitable accounts. Flat per-task pricing on a skewed cost distribution is structurally vulnerable to adverse selection. The customers who stay are disproportionately the ones who cost you the most.

The "stable mix" assumption fails hardest exactly when you scale. Early on, your task distribution is narrow because you have few customers and similar use cases. The blended rate looks reliable, which builds false confidence. As you grow into new verticals, the distribution fattens and shifts, precisely when you've raised money on those early margins and locked in flat-rate enterprise contracts. The metric is most trustworthy when the stakes are lowest and least trustworthy when they're highest.

Caching corrupts the blend in a sneaky direction. Prompt caching and memory reuse cut cost dramatically, but unevenly. Repetitive easy tasks cache beautifully; novel hard tasks barely cache at all. So caching pushes your median down far faster than your tail, which means it widens the spread between cheap and expensive tasks and makes a single blended number even less representative than before. A caching win can make your margin reporting less accurate while genuinely improving the business.

A blended rate hides which customers to fire. Because the average smears cost across the whole base, it never tells you that 5% of customers might be consuming 40% of your model spend. Per-customer cost-to-serve routinely surfaces accounts that are individually unprofitable at any plausible price, the kind you should reprice, cap, or offboard. The blended rate is the reason most GaaS teams don't know those customers exist.

References

• The Cost of Cloud, a Trillion-Dollar Paradox, Andreessen Horowitz (a16z)
• Anthropic Model Pricing
• The Economic Potential of Generative AI: The Next Productivity Frontier, McKinsey & Company