TokenOps: why a 40-turn agent costs 16×, not 4×

A task your agent finishes in 10 turns costs you 1×. Let it run 40 turns and you'd expect 4×. It's 16×. Every turn re-sends the context that the earlier turns built up, so uncached input cost grows with the square of the turns, not in a straight line. The fix is to engineer the context so most of it is read from the cache, at a tenth of the price.

That's the opening of TokenOps, a four-minute talk by Gareth Bland, chief data scientist at Microsoft Frontier Company. He defines TokenOps as the data science that lets a company keep deciding where its tokens should go: where they move the needle most. Short talk, three ideas worth keeping.

The price list

Model providers don't charge one price per token. Taking your input messages as 1×, this is the scale he showed:

Token	Cost	What it is
Output	5×	Where reasoning happens, the most useful thing an agent does
Base input	1×	The messages you send
Cache write	extra	Paid as the loop writes context to the cache
Cached input	0.1×	Context read back from the cache

So the expensive part is also the valuable part: reasoning is output. The part you control is how much of the input is cached.

Outputwhere reasoning happens
5×
Base inputthe messages you send
1×
Cache writepaid as the loop writes to the cache
extra
Cached inputread back from the cache
0.1×

Yours to controlHow much of the input comes from the cache.

The price scale he showed, with your input messages as 1×. Output, base input and cached input are drawn to scale. The talk gave no multiplier for cache writes, so that bar is only marked, not measured.

Loops make it quadratic

Agents don't answer in one shot anymore. They take turns: act, look at the result, reason, act again. Each turn carries everything that came before.

Turns	You'd expect	Uncached input actually costs
10	1×	1×
40	4×	16×

Four times the turns, sixteen times the cost. Context engineering is how you bring that curve down: keep the stable part of the context in front and unchanged, so the provider can serve it from the cache at 0.1×.

You'd expect

4×

Four times the turns

Every 10 turns cost the same

Uncached input actually costs

16×

Each turn re-reads everything before it

Four times the turns, sixteen times the cost

The fixKeep the stable part of the context in front and unchanged, so it's read from the cache at 0.1×.

Drawn to scale, turns 0 to 40 left to right. The height at each turn is the context it re-sends, so the area is what you pay for uncached input. The ink triangle is the first 10 turns, 1×. Forty turns is sixteen of them, not four.

Did the tokens buy anything?

Cost alone says nothing about whether the spend was worth it. Bland scores agent work with an agentic efficiency score made of three ratios, read as a funnel from all the work an agent starts down to what users feel:

Autonomous completion

Work finished without failure or a human stepping in.

Merged to production

Token efficiency

Of the finished work, what actually shipped.

Serving users

Value generation

Of what shipped, what moved the needle.

The third ratio is the one a token dashboard can't give you. My own Meterlex prices every model call from the session logs, which covers the cost side. It can't tell me whether a session's work got merged, let alone whether anyone used it. That needs data from the repo and the product, not the model provider.

The S-curve: when to stop spending

The last idea is about where a product sits. Features pay off on an S-curve, and Bland names three zones:

Zone 1

Underinvestment

Not enough features yet for an MVP or a pilot.

Threshold

Zone 2

Leverage

Every new feature or improvement brings value back to users.

Saturation

Zone 3

Gold plating

Polishing what exists, adding what nobody asked for.

Agents make zone 3 cheap to enter. When a feature costs a prompt instead of a sprint, nothing stops you gold plating except knowing where you are on the curve. That's his closing point: in agentic engineering, knowing where your team or your next feature sits on the S-curve matters most.

What I take from it

Measure the curve, not the bill. A monthly total hides the quadratic part. Turns per task and the cached share of input say where the money goes.
Cost is the top of the funnel. Token spend only means something next to how much of the work merged and how much of it users felt.
Spend where the curve is steep. The cheapest token is the one you don't spend gold plating.

1Measure the curve

Turns per task and the cached share of input, not the monthly bill
2Cost is the top

It means something next to what merged and what users felt
3Spend where it's steep

The cheapest token is the one you don't spend gold plating

What I'd watch after this talk: the shape of the cost, the funnel under it, and where a feature sits on the S-curve. The dashed top of the last curve is gold plating.

Plain text: tokenops-agent-cost-curve.md · Atom feed · All notes