Measuring the Marginal Value of an Extra Agent Run
Every additional agent run costs you something and returns something, but the two curves almost never move together. The marginal cost of one more run is roughly flat; the marginal value falls off a cliff. The job of any serious GaaS operator is to find the run number where the value of attempt N+1 drops below what it costs to produce, and to stop selling (or charging for) runs past that point. This piece shows how to measure that crossover with real math, not vibes, and why most agent products are quietly torching margin by running well past it.
Table of Contents
- Why Marginal Thinking Beats Average Thinking
- Defining the Marginal Value of an Agent Run
- The Two Curves: Marginal Cost vs. Marginal Value
- A Worked Example: The Support Agent at Run Five
- Where the Value Actually Comes From
- How to Instrument This in Production
- Pricing Implications: Who Pays for the Low-Value Runs?
- Common Mistakes When Measuring Marginal Run Value
- Insights Most People Overlook
- Frequently Asked Questions
- Conclusion
- References
Why Marginal Thinking Beats Average Thinking
Most dashboards in agentic AI report averages. Average cost per task, average success rate, average tokens per run. Averages are comfortable because they're stable and they make a quarterly slide look tidy. They're also where a surprising amount of margin goes to die.
The problem is that an agent rarely solves a task in a single, clean pass. It retries. It re-plans. It calls a tool, gets a bad result, and tries a different tool. A customer-support agent that "resolves 80% of tickets" might do so by running the same ticket through three, five, or in pathological cases fifty model calls before it lands an answer, a dynamic worth understanding in its own right, since the hidden cost of retries compounds fast. When you only look at the average across all those runs, you can't see that the first run did 70% of the useful work and the next four did the remaining 10% at full price.
Marginal analysis asks a sharper question: what did the last run buy me? Economists have leaned on this for two centuries because it's the only honest way to decide whether to do one more of anything. The marginal-cost, marginal-revenue framing, produce until the cost of the next unit equals the revenue it brings in, is the same logic a factory uses to decide its production volume, and it maps almost perfectly onto the decision of whether to let an agent take another swing. Investopedia's primer on marginal cost and marginal revenue is a clean refresher if the terms have gone fuzzy.
The twist for GaaS is that, unlike a factory, the "production volume" decision is being made hundreds of times per second, automatically, by software you may not fully control. So you'd better know where the line is.
Defining the Marginal Value of an Agent Run
Let's pin down terms, because sloppiness here is where the analysis falls apart.
A run is one bounded attempt by an agent to make progress on a task, one autonomous loop that ends in a result, a hand-off, or a give-up. Definitions vary by platform, and you should define yours explicitly, because it anchors everything downstream (this is closely tied to defining cost-per-completed-task as the category's core unit).
Marginal cost of run N is the all-in cost of that specific run: input and output tokens, any reasoning or "thinking" tokens, tool-call fees, retrieval and memory reads, and an allocated slice of fixed infrastructure. Crucially, this includes the cost of runs that fail, because a failed run still burns compute.
Marginal value of run N is the expected incremental business outcome that run produces, on top of everything runs 1 through N-1 already delivered. For a support agent, that might be the probability that run N resolves a ticket that earlier runs didn't, multiplied by the dollar value of a resolution. For a coding agent, it's the probability run N produces a passing diff times the value of that diff.
The word doing the heavy lifting is incremental. The value of run 1 is large because it's measured against a baseline of zero. The value of run 6 is measured against a baseline of "five runs already tried and got close," so even if run 6 succeeds, the marginal lift over what you already had may be tiny, the task was 90% solved already.
This is just diminishing marginal returns, the same curve that governs fertilizer on a field or engineers on a project. Each additional unit helps less than the one before it. What's specific to agents is how steep the drop-off is and how invisible it is without instrumentation.
The Two Curves: Marginal Cost vs. Marginal Value
Picture two lines on the same chart, with "run number for a given task" on the x-axis.
The marginal cost line is roughly flat, maybe gently rising. Each run costs about the same in tokens and tool calls. It can actually creep upward on later runs, because agents tend to stuff more context, prior attempts, error messages, accumulated memory, into each subsequent prompt, and longer context means more tokens. So if anything, late runs cost a hair more.
The marginal value line starts high and decays fast, often something close to exponential decay. Run 1 captures most of the easy wins. Run 2 mops up a meaningful chunk. By run 4 or 5 you're scraping at the genuinely hard residual cases, where each additional attempt has a low probability of cracking what the previous attempts couldn't.
The two lines cross somewhere. Call it run N*, the break-even run. Past N*, every additional run costs more than the expected value it returns. You are, in a literal economic sense, destroying value with each attempt beyond that point.
The entire discipline of measuring marginal run value comes down to finding N* for each task type and then deciding what to do at the boundary: stop, escalate to a human, or hand back a partial result. This is where the abstract metric meets the human-intervention rate as a churn signal, a well-placed escalation at N* is often worth far more than a sixth autonomous swing.
A Worked Example: The Support Agent at Run Five
Numbers make this concrete. Suppose a customer-support agent and a resolved ticket worth $8 to the business (deflected human-agent time, retained goodwill, whatever your finance team blesses). Each run costs $0.12 all-in.
Now suppose the probability that a still-unresolved ticket gets resolved on each successive run looks like this:
- Run 1: 55% chance of resolution
- Run 2: 25% chance (of those still open)
- Run 3: 12%
- Run 4: 6%
- Run 5: 3%
The marginal value of each run is the probability the ticket reaches that run still unresolved, times the conditional resolution chance, times $8.
- Run 1 reaches 100% of tickets → marginal value ≈ 0.55 × $8 = $4.40, cost $0.12. Wildly worth it.
- Run 2 reaches the 45% still open → 0.45 × 0.25 × $8 ≈ $0.90. Still a no-brainer.
- Run 3 reaches ~34% still open → 0.34 × 0.12 × $8 ≈ $0.33. Fine.
- Run 4 reaches ~30% → 0.30 × 0.06 × $8 ≈ $0.14. Now we're close to the $0.12 cost line.
- Run 5 reaches ~28% → 0.28 × 0.03 × $8 ≈ $0.067. Below cost. You're lighting money on fire.
N* sits between run 4 and run 5. After run 4, the right move isn't run 5, it's escalation to a human or a graceful "I've opened a ticket for our team." The agent that mechanically grinds to run 8 because someone set max_iterations=8 is converting margin into nothing.
Two things make this example more than a toy. First, the resolution probabilities are empirical, you can measure them from your own logs, you don't have to guess. Second, the dollar value of an outcome is a number your business can defend, which is exactly what's required to make the agent ROI claim verifiable rather than aspirational.
Where the Value Actually Comes From
Run value isn't a single quantity; it decomposes, and the decomposition tells you where to intervene.
Resolution value is the headline: did this run finish the job? But there's also partial-progress value, a run that doesn't finish the task but advances it (gathers the right context, narrows the problem, sets up the next run to succeed). On long-running, multi-step agents this matters a lot, because the "value" of an intermediate run is its contribution to an eventual success several runs later, not an immediate outcome. Attributing value across a chain is genuinely hard and it's where naive per-run accounting breaks.
There's also negative marginal value, which most teams never measure. A run can actively make things worse: it hallucinates a wrong answer that the customer acts on, it writes code that introduces a bug, it spawns sub-agents that fan out into a cost spiral. When you price the value of a run, the expected-value calculation has to net out the probability-weighted cost of these bad outcomes. A run with a 3% chance of a $4 win and a 2% chance of a $20 mistake has negative expected marginal value even before you count its compute cost.
McKinsey's analysis of where companies actually capture value from generative AI keeps returning to the same point: the gains concentrate in a few high-value workflows and evaporate when deployments chase marginal use cases, a macro echo of the same diminishing-returns curve we're tracing at the per-run level. Their ongoing work on the economic potential of generative AI is worth reading with this lens.
How to Instrument This in Production
You cannot manage what you don't log at the right grain. Average-level telemetry is useless here. You need per-run, per-task event data that lets you reconstruct the sequence.
Log, for every run: the task ID it belongs to, the run index within that task, the all-in cost, the outcome (resolved / progressed / failed / errored), and a confidence or verification signal. With that, you can build the empirical resolution-probability curve per task type, which is the raw material for finding N*.
Then compute the marginal value curve by cohorting tasks: of all tasks that reached run 3, what fraction resolved at run 3 versus earlier? Multiply by your outcome value. Plot it against the cost curve. The crossover is N*. Recompute it monthly, because model upgrades, prompt changes, and tool reliability all shift the curve, sometimes dramatically. A better base model can push N* down (more value captured early, less need for late runs) even as it makes each run cheaper, which is a delightful double win that average dashboards completely hide.
This per-run discipline is the same data backbone that powers a proper GaaS metrics dashboard every operator should track; marginal run value is one of the highest-leverage metrics it can surface. If you're building the dashboard anyway, instrument for marginality from day one rather than retrofitting it.
A practical guardrail: set max_iterations per task type from the data, at or just past N*, instead of picking a round number like 10 and forgetting about it. And add a dynamic stop, if a run's verification confidence isn't improving across attempts, halt early rather than burning the remaining budget.
Pricing Implications: Who Pays for the Low-Value Runs?
Here's where the economics get political. If you charge customers per run, you have a perverse incentive to let agents run past N*, every wasted run is revenue. That's a short-term win and a long-term trust catastrophe; customers eventually notice they're paying for fifty model calls to answer one question. Per-run pricing also makes the marginal-value problem the customer's problem, which is a fast way to lose them.
If you charge per outcome (per resolved ticket, per merged PR), the marginal-cost of low-value runs lands squarely on you, the vendor. Now you have a sharp incentive to find N* and stop, your margin literally depends on it. This is one underappreciated reason outcome-based pricing is gaining ground in GaaS: it aligns the vendor's economic interest with actually knowing where run value crosses run cost. Andreessen Horowitz has argued that this shift toward charging for work delivered rather than seats or tokens is what makes agent businesses defensible, and marginal-run discipline is the operational core of pulling it off.
The uncomfortable middle ground is flat-rate or subscription pricing, where the vendor eats all marginal run cost with no per-use revenue to offset it. There, ignorance of N* doesn't just shave margin, it can invert it on heavy-usage accounts. The runaway-usage customer who triggers eight runs per task on a flat plan is the GaaS equivalent of the gym member who actually shows up every day.
Common Mistakes When Measuring Marginal Run Value
A few errors show up again and again, and each one quietly corrupts the number.
Measuring average value and calling it marginal. The single most common mistake. Total value divided by total runs tells you nothing about whether run 6 was worth it. You need the incremental contribution of each run index, conditioned on reaching it.
Ignoring failed and errored runs in the cost base. Failed runs cost real money. If you only count the cost of successful runs, your marginal cost is artificially low and N* looks further out than it is. Every burned token counts, win or lose.
Treating outcome value as a fixed constant. The value of resolving a ticket isn't uniform, a churning enterprise account's ticket is worth more than a free-tier user's. Segment your outcome values, or your N* will be wrong for both ends of the distribution.
Forgetting negative outcomes. As covered above, a run can have negative expected value. Teams that only model upside systematically overestimate how far it's worth running.
Setting max_iterations by superstition. Round numbers chosen in a planning meeting have no relationship to your actual cost/value crossover. Let the data set the cap.
Recomputing N* never. The curve moves every time you change a model, a prompt, or a tool. A stale N* from two quarters ago is just a guess wearing a lab coat.
Insights Most People Overlook
1. The most valuable "run" is often the one you don't take. Past N*, the highest-EV action isn't another autonomous attempt, it's a clean escalation. Counterintuitively, a vendor's run-efficiency improves when it gets better at giving up at the right moment. Knowing when to stop is a feature, and almost nobody markets it as one.
2. Marginal value can go negative, and that ceiling is lower than the cost ceiling. Most operators implicitly assume the worst case for an extra run is "wasted compute." It's not. A confidently wrong late run can cost a customer relationship worth thousands. The stop-point set by risk is often earlier than the stop-point set by cost, and far earlier than the one set by max_iterations.
3. A cheaper model can make late runs more tempting and less worthwhile at the same time. When per-run cost drops, the naive instinct is to allow more runs, the cost line fell, so N* must move right. But a better model usually also captures more value early, which steepens the value-decay curve and pushes the real N* left. Falling per-run cost and rising per-run capability move N* in opposite directions; you have to actually measure which dominates.
4. Fan-out quietly multiplies the marginal cost of a single "run." When an agent spawns sub-agents, one user-visible run is secretly a dozen model calls. Your marginal-cost line isn't flat, it's a step function that jumps every time the agent decides to delegate. If your run accounting doesn't roll up sub-agent spend into the parent run, N* is a fiction.
5. The crossover point is a competitive moat, not just a cost control. A vendor who knows N* precisely can price per-outcome aggressively, undercut competitors who are eating low-value runs blind, and still protect margin. The marginal-value curve is proprietary operational knowledge, it lives in your logs and nobody else's. Treat it like the asset it is.
Frequently Asked Questions
How is marginal run value different from cost-per-completed-task? Cost-per-completed-task is an average over an entire task (all its runs). Marginal run value is the incremental contribution of one specific run within that task. You need both: the average for pricing and reporting, the marginal for deciding when to stop.
Do I need per-outcome pricing to benefit from this analysis? No, but the incentives are cleanest there. Even on flat-rate or per-seat plans, knowing N* directly protects your gross margin, arguably it matters more under flat pricing, where you eat every wasted run with no offsetting revenue.
How do I value a run that makes partial progress but doesn't finish the task? Attribute a fraction of the eventual outcome value to it, based on its contribution to a later success, typically via cohort analysis of which intermediate states lead to resolution. It's harder than measuring terminal runs, but ignoring partial progress badly understates the value of early runs in long chains.
What's a reasonable starting max_iterations before I have data?
Start conservative, three to four for most task types, and widen only where logs show late runs still clearing the cost line. Starting high and trimming down means you pay for the low-value runs while you learn.
Does this apply to long-running agents that work for hours? Yes, but the unit shifts. For long-running agents the relevant "run" may be a checkpoint or a phase rather than a quick loop, and marginal value accrues over a much longer horizon. The same cost-vs-value crossover logic holds; you're just measuring it on a different clock.
How often should I recompute the break-even run? Monthly at minimum, and immediately after any model swap, major prompt change, or tool reliability shift. Each of those moves the curve, and a stale crossover point silently misallocates spend until you refresh it.
Conclusion
Measuring the marginal value of an extra agent run is the per-run sibling of every other metric in agent economics: cost-per-completed-task gives you the average, success rate gives you reliability, and marginal value tells you when one more attempt stops being worth it. The core move is simple to state and hard to do well, build the empirical value-decay curve per task type, lay the flat-ish cost curve over it, find the crossover N*, and engineer your agent to stop, escalate, or hand off there rather than grinding on by default.
The operators who win at GaaS won't be the ones whose agents run the longest. They'll be the ones who know, to a fraction of a run, exactly how long is worth it, and who treat that crossover point as the proprietary, margin-defining asset it is. Get the instrumentation right, recompute as your models evolve, and let the data, not a round number in a config file, decide when the next run is one too many.
References
More in Economics
- The Free-Tier Trap: Why AI Agents Are Too Expensive to Give Away
- Seat-to-Task Revenue Conversion: How to Model the Transition Without Blowing Up Your Forecast
- Discount Dynamics: Volume Pricing When Your Costs Refuse to Sit Still
- Agent Utilization Rate: The Quietly Decisive Metric in GaaS Economics
- Why GaaS Valuations Can't Use SaaS Revenue Multiples