Review the plan, not the pull request

Overmind stopped reading its own pull requests. Almost all the code it ships to production is now read by no human, the author included. Review didn't go away, though. It moved to a plan of about a page and a half that people argue over before an agent writes a line, and in the three weeks after the switch the team closed three times as many issues while lead time fell from 3.2 days to 1.3.

The talk is Pull Requests Are Dead, Long Live Peer Review by Dylan Ratcliffe, founder and CEO of Overmind, from AGNTCon + MCPCon North America 2026. Overmind uses AI to simulate changes to large back-end infrastructure. In my software factory note, the slow part of my own setup is my review queue. This talk is one team's answer to that.

Review was doing four jobs

Like most teams, Overmind leaned on code review for much more than bugs. It caught bugs, it covered SOC 2 and ISO 27001 compliance, it trained junior engineers, and it built a shared picture of how the system actually works.

What made it work is that a human cared about every line. Comment on a person's pull request and something good happens either way. If you're right, they learn and the product gets better. If they push back, there's a constraint you didn't know about, and now you do.

Comment on a pull request from a bot that addresses its own review comments, and you get the same eager agreement whether you're absolutely right or absolutely wrong. That's his first lesson: AI is not your peer, so you can't peer review its work.

Two people out-shipped the company

At the start of the year Overmind was about 80% engineers. Product was Dylan and one product manager. They came up with the idea, did the research and worked out the shape, meaning the high-level architecture. Then came a handover meeting, a round of questions, and engineers planned and built it with AI.

The catch: once Dylan had done everything up to the shape, handing it straight to an agent was faster than handing it to a person. He lives in San Francisco, a bunch of his engineers are in Europe, and the agent never sleeps.

So one week he and the product manager produced more than every other person in the company combined, on every metric they had: lines of code, pull requests, tickets closed and story points. That's only six people, but it's two people doing the work of six full-time engineers. "Neither of us are engineers," he said.

He's open that he seriously considered firing half the engineers and replacing them with product managers. He didn't. He flew everyone to France for a week to sort out the process instead.

The work moved left

This is the slide I grabbed from the talk, redrawn. Before, product owned the idea, the research and the shape, and engineering owned the plan, the build and the ship. Now product keeps the idea, engineering does the research, the shape and the plan, and an agent builds and ships.

His second lesson came out of that week: if you think your job is turning detailed specs into high-quality code, you're out of a job. He says that's been true since January and Opus 4.5.

One page, 24 hours, no questions

Moving engineers left came with a rule. Product hands over a one-page doc. The engineer who gets it shows a demo within 24 hours, and can't ask a single question in that time.

A one-page doc is full of gaps, and that's the point. Does this need to work on mobile? Go and research whether people use this kind of app on mobile. Forced to settle the ambiguity themselves, the engineers were mostly right.

Nobody can absorb all that new ambiguity and still care about every line of code, though. So the line-by-line detail went to agents.

Not every agent loop is equal

When they studied how people build agentic "software factories" and "dark factories", most ran what he calls a do-while loop. The agent picks up a ticket, decides the approach, makes the trade-offs, writes code and tests, maybe records a demo video, and opens a pull request at the very end. Then a human has to reverse-engineer every decision from the code.

He's a little worried about AI taking his job. He's more worried about AI taking the interesting part and leaving him paid to sign off on decisions he wasn't in the room for.

So they built an until loop. Agents research the codebase, for example to check whether there's already a practice for this kind of change. Humans decide the approach, make the trade-offs and write an implementation plan, and a peer reviews it. Then an agent writes the code and a second agent compares it with the plan. Any difference sends it back, and it loops until code and plan match.

Once the loop looks like that, the place for peer review is obvious: before the loop starts.

Review the plan, then stop reading

Plans get real review because people care about them again. When someone comments on Dylan's pull request today, his honest reaction is "I didn't even read that code. If anything, you're the first one to have read it." When someone comments on his plan, he argues back, because he wrote every line and knows why each one is there.

He treats an approved plan as a pre-approved change. Most compliance setups already know that idea: a well-defined change can be approved in advance, as long as you do exactly that. The loop can't exit until the code matches an approved plan, so the code goes to production without a second human review.

Two checks keep it honest:

• Humans sample a small percentage of the plan-versus-code checks, to make sure the review agent isn't getting them wrong.
• If the coding agent finds a better idea, or a real reason it can't follow the plan, it records a difference, and a human acknowledges it.

Overmind was being audited when he gave the talk. He strongly believes the process is compliant, and he'll know when the audit ends.

The tooling has no user interface at all. Plans get written in Claude Code or whatever harness someone uses, and sending a plan, pulling it down and leaving a review are all MCP calls.

Three weeks later

They changed the process in one week, then measured the next three weeks against the three before. Issues completed tripled, and so did commits. Lead time, from a ticket being picked up to running in production, fell from 3.2 days to 1.3.

These are one small team's numbers, three weeks against three weeks, measured by the company itself. And Overmind now sells a product built on this exact idea, so take the talk as a strong argument from someone with a stake in it.

Attention was the bottleneck

They were never spending 90% of their day on code review, so the 3x didn't come from skipping review time. His answer is attention. Pull requests kept too many balls in the air and forced constant context switching.

The old timeline for one change: about 3 hours writing code, 15 minutes opening the pull request, a day and a half waiting for a busy reviewer, then an hour of back and forth. You can't forget any of it, because you're the only one who understands the code. By the time a comment came back, he'd written 15,000 more lines and couldn't remember what he was trying to do, so abandoning the change could be easier than switching back into it. About four and a half hours of work lived in his head for about 40 hours.

A plan is faster to write and much faster to review than 3,000 lines of Go. Once it's approved, you commit it to memory as how the system works. The code doesn't exist yet, but it will in about three hours, because there's no other way out of the loop. That costs a couple of hours of attention.

A recipe is easier to reject than a meal

Standing in the kitchen deciding what to cook, it's easy to say "I don't feel like salad." Once the salad is made, you can't. Telling someone to cook a meal again takes Gordon Ramsay energy, and that's what rejecting a pull request is. It's emotionally hard, which is why it was usually only the most senior engineers who did it.

Plans changed that. Overmind is now 2.8 times more likely to reject a plan than it ever was to reject a pull request, and the rejections come from everyone on the team, product included. It's also far cheaper: a rejected plan adds about 40 minutes of lead time, while any comment at all on a pull request added 16 hours.

His big worry going in was agents writing the plans and agents reviewing them, with nobody reading anything. He says the numbers show it didn't happen. Plans are short enough that people read them, and they care about them.

It also gives juniors a way in. A junior who doesn't follow a plan can talk it through with their agent until they do, and reject whatever still doesn't make sense. Dylan's product manager reviews engineering plans exactly like that, and when he pushes back, he's right about 80% of the time.

Dylan counts the 8% of plans they abandon as a win. The what looked good, the how showed it wasn't worth it, and nobody had to build it just because it was already in the sprint.

The how still matters

He closed by disagreeing with a Paul Graham line: "When anyone can make anything, the big differentiator is what you choose to make." Dylan says the differentiator is still how you make it.

Two laptops can share the exact same spec sheet and be completely different products. The difference is hundreds of how decisions along the way, and those decide whether a product performs, scales and stays easy to debug. His advice: don't sit at the bottom of a loop where the agent makes those calls for you, and say no to tools that put you there.

What changes is the layer you answer them at. You stop caring how the files are organized or what the functions are called. You still decide how data flows, how authentication works and how the system handles failure.

Try it

Overmind published the internal tool as until, "git-backed tickets for people and agents". You write the plan, someone signs it off, each section of the plan becomes a job your agents claim, and a review agent compares the code with the plan, so anything missing or extra comes back to you as a decision. It's waitlist-only right now, and the site lists Claude Code, Cursor, Codex, Pi, Factory Droid, Amp and OpenCode.

The argument also exists as a manifesto, AI Is Not Your Peer, by Dylan Ratcliffe, James Lane and David Schmitt. Its key line: "The artifact worth reviewing is the plan we wrote before the agent wrote anything."

If you want more:

• Pull Requests Are Dead, Long Live Peer Review, the talk, 25 minutes
• AI Is Not Your Peer, the manifesto
• until, the plan-review tool, on a waitlist
• The agentic software factory is the 2022 one with one box swapped, where review became my bottleneck
• What breaks when an AI agent runs a real business, another AGNTCon + MCPCon talk

Watch the talk, then try reviewing your next agent task as a plan before any code exists. Tell me if your lead time moves.

Plain text: review-the-plan-not-the-pull-request.md · Atom feed · All notes