--- title: Fork your agent's harness, keep the winner description: Everything around a coding agent that you can change is just code and text. At AGNTCon + MCPCon Europe, Fausto Albers showed how his team improves it. The eval sits outside the loop, cost per accepted change says when a busy loop has stalled, and a meta agent runs several harness variants on the same task and keeps the best. date: 2026-10-11 tags: AI agents, harness engineering, evals, Claude Code, agentic engineering --- Everything around a coding agent that you can change is just code and text: the CLAUDE.md, the subagents, the skills, the tools, the memory policy. So when your agent loop stalls, don't guess one fix. At AGNTCon + MCPCon Europe in Amsterdam, Fausto Albers showed his team's move: a meta agent writes several versions of the harness, runs them all on the same task against an eval that sits outside the loop, and keeps the one that wins on whatever you count as cost. The losers get stored too. He opens with the irony. Attention is all you need, and now attention is the scarce resource. Since 2022 the work he can hand to AI has grown enormously, and his attention didn't follow that curve. His agents make thousands of decisions toward goals he gives them, and he isn't there for most of them. So the question of the talk: how do you build a system that knows what you find important, and what you count as a cost? ## A steel beam, drawn in 2D Fausto runs WonderWhy, which works closely with the applied AI lab for industrial digital twins at the Amsterdam University of Applied Sciences, and he co-founded AI Builders Club in Amsterdam. He has a background in sociology and used to own cocktail bars and restaurants. That's how he got into tech: a startup using machine learning to predict the flavor preferences of hospitality customers. His case came from a steel company. Its CNC machines know what to do once they get directions compiled from a valid 3D model. But a lot of its clients never get a 3D CAD model. They get a 2D PDF drawing of the beam, a lossy copy of the 3D model, and a trained engineer works out the rest by hand. Only months ago, turning that drawing into a 3D model wasn't possible, and his team got stuck. So they interviewed the engineers. Each one had the PDF on the left and a CAD program on the right, and rebuilt the beam step by step: making mistakes, correcting them, building a spatial model of the part in their head. That's exactly what they let the agents do, and it works. They delivered it in September. It still costs some money to run, and they're training an open-source model on the Dutch supercomputer so it'll cost almost nothing.

What the client sends

2D PDF

A lossy copy of the 3D model

Takes a trained engineer to read

What the CNC machine needs

3D CAD

Directions compile from a valid 3D model

Rebuilt step by step, mistakes and all

The loopEngineers: PDF on the left, CAD on the right. Now the agents run that same loop.

The steel case, drawn for this note rather than taken from the client's files. Keep an eye on the small symbol at the top right of the drawing: it comes back later.
Then the line that sets up the rest of the talk: his team didn't write a single line of that application. They designed the system that built it. ## The harness is the part you can edit His short history of the last few years: prompt engineering, then context engineering, and now it's about designing the harness and running the loop. He's precise about what "harness" means here. Two things he can't change: the foundation model, and the outer harness, meaning the coding environment, Claude Code or Codex or whatever you run. Everything inside that is his to change, and it's just code and text: - the instructions, CLAUDE.md and AGENTS.md - the subagents, their roles and how they coordinate - the context - the memory policy ("there's no learning without memory") - the skills and functions That's a lot of knobs. Set them up differently and the same task comes out different. And in practice, he says, agents already build these harnesses too.

Can't change

Foundation model

Coding environmentClaude Code, Codex…

Can change · just code and text

  • InstructionsCLAUDE.md, AGENTS.md
  • Subagentstheir roles
  • Coordinationhow they work together
  • Contextwhat each one sees
  • Memory policyno learning without memory
  • Skills and functionswhat they can do

Same taskTurn the knobs differently and the output comes out different.

His split of the harness, at 6:29 in the talk. The memory policy is marked because of his line about it.
## Keep the judge outside the loop Around the harness runs the loop: act, check the result, go again. The check can't live inside the harness. The more capable the models get, the better they get at reward hacking, so the eval has to sit outside the system. It can be an agent or plain deterministic code. What matters is that it isn't the thing being graded. The feedback lands in files on disk, and those files are the input to the next iteration. So every turn of the loop can start from a fresh context window instead of an ever longer conversation. That matters for cost too: my [TokenOps note](/notes/tokenops-agent-cost-curve.html) is about how a long agent conversation's input cost grows with the square of its turns.

The system being improved

Outside it

1 Act

Harness

Runs the task. The part you can edit.

2 Check

Eval

An agent or deterministic code. Never the thing being graded.

3 Remember

Files on disk

The feedback, saved as files.

The loop he described. The harness and its files sit inside the dashed line; the eval sits outside it, because the more capable the model, the better it gets at gaming a check it can reach.
## Watch the cost per accepted change The number he finds most useful is the cost per accepted change. Each iteration costs something, and the eval is your proxy for progress toward the goal. When the cost keeps rising and the progress stalls, you've hit diminishing returns. The loop still looks very busy. Your credit card is burning. That's the moment to stop and ask what the harness is missing, because it's just code and text you can change. And cost doesn't only mean euros. If a client has a six-figure budget and time is the problem, spending a lot of tokens can be the sensible call.
Cost per accepted change, schematic Over the loop's iterations, progress rises fast and then flattens, while the cost per accepted change stays low and then climbs. Past the stall is the zone of diminishing returns: the loop looks busy and the card is burning.

When it bendsStop and ask what the harness is missing.

The shape he described at 8:47, drawn as a schematic with no data: the curves are illustrative, not measured. The square marks where progress stalls.
## Three harnesses, one task Here's the part I'd steal. When a loop stalls, the usual move is to think about what could be different, write one new version of the harness and try it. Fausto's team hands that to a meta agent instead: one agent, or a bigger system, that reads every signal the coding harness produces. The meta agent doesn't write one new harness. It writes three, or many. One gets different instructions, one changes the order of work, one adds or removes a tool. Each version runs the same task, the eval scores it, and you compare outcome and cost against the original. Every version goes into a database as a snapshot of the coding environment. The one that does best, by whatever you find important, becomes the new harness. The others stay in the database with their evals and traces, because they might come in handy: what fixed a problem in one project might fix one in the next.

Meta agentreads the signals, writes the variants

  1. Originalthe harness as it is

    stored

  2. Variant Adifferent instructions

    stored

  3. Variant Bdifferent order of work

    kept

  4. Variant Ca tool added or removed

    stored

Snapshot database

Every version, its eval and its traces. The losers too.

Same taskOutcome and cost, where cost is whatever matters to you right now.

A schematic of the variant step, at 10:01 in the talk. Which variant wins here is arbitrary; in his telling, it's the one the eval rates best on what you care about.
In the Q&A someone asked about prompt optimizers like GEPA. He's used them in MLflow on another project, and DSPy, and calls all of it a form of reinforcement learning, of evolutionary algorithms. For more, he pointed to Meta's [Hyperagents](https://arxiv.org/abs/2603.19461) paper, which builds on the [Darwin Gödel Machine](https://arxiv.org/abs/2505.22954), and to [JIT-Agent](https://arxiv.org/abs/2608.25593), which trains an open-source model to generate harnesses. He uses that one as well. ## The symbol nobody put in the harness The example that made it concrete came from the steel project. One harness gave the pipeline, version A, a way to crop the technical drawings and zoom in on details. What it didn't know was that some very small symbols on those drawings are very important. That knowledge wasn't in the harness. The evals stalled. That's a signal, but it doesn't tell you what's wrong or what to change. They explored many directions and landed on a database of those symbols. The next harness gave the pipeline a way to look them up, and that became the policy from then on.

  1. 1Version A

    Crops and zooms into the drawing. Doesn't know the small symbols matter

  2. 2Evals stall

    A signal, not a diagnosis: it doesn't say what's missing

  3. 3Look it up

    The next harness can look the symbols up in a database

After thatThe lookup became the policy for the harnesses that followed.

The symbol story, at 11:59 in the talk, drawn for this note: the small mark at the top of the first drawing is the kind of detail version A never looked at.
## Cost is a state of mind Keeping every version raises the next problem. With a thousand harnesses in the database, how do you find the one that fits a new situation? His answer leans on how people remember. We don't recall by piling up detail, and the systems we build work very differently from human recall. He thinks we can learn from that. His example is the same situation, handed to two agents, human or AI. One knows money isn't the problem and there's enough time. It can afford to be curious. Curiosity is expensive, learning is expensive, and it can pay off big. The other knows the project has to ship. There's no room to explore anymore: things just need to be checked. He's careful here. He doesn't say an AI has emotions. He says states of mind like curiosity, concern or urgency can stand in for what we count as cost. Today a coding agent with a goal is barely aware of what's important to you. It just tries to complete the goal against some eval.

Money's fine, there's time

Curious

It can afford to explore and try things

Learning is expensive, and it can pay off big

The project has to ship

Urgent

No room to explore anymore

Things just need to be checked

Not emotionsA stand-in for what you count as cost. Today's coding agents rarely know it.

His two agents in the same situation, at 14:00 in the talk. Same task, different sense of what's expensive.
## Where my own harness sits He ended on four things to take home: let useful methods outlive the task, think hard about what costs mean because it's a lot more than tokens, let competing perspectives shape the next move, and learn. My own setup is the inside of his diagram. A CLAUDE.md per folder, 26 skills of my own, a hook that refuses to open a pull request without an issue number, and a memory folder of about 200 one-fact files. When I correct an agent, it saves the lesson with a why, and the next session reads it. So part of my harness is already written by agents. But every change is one variant, kept because I said so. No snapshots, no two versions racing on the same task. Reviews judge each code change; nothing judges the harness itself. That's the single-hunch habit Fausto's team moved past.

1 · me, today

One fix, by hand

A rule written after a mistake, kept because I said so.

An eval outside

2

A judged loop

Every run scored from outside, cost per accepted change in view.

A meta agent

3 · Fausto's team

Variants, keep the best

Several harnesses on the same task, every snapshot stored.

The number I'm missing is his cost per accepted change. [Meterlex](/projects/meterlex.html) already prices every session I run, but nothing joins that to which changes got merged. The [talk](https://www.youtube.com/watch?v=kic8ijrzxLI) is 18 minutes. The harness split is at [6:29](https://www.youtube.com/watch?v=kic8ijrzxLI&t=389s), the variants at [10:01](https://www.youtube.com/watch?v=kic8ijrzxLI&t=601s) and the symbol story at [11:59](https://www.youtube.com/watch?v=kic8ijrzxLI&t=719s). If you've raced two versions of a CLAUDE.md on the same task, I'd love to hear which one won, and why.