What breaks when an AI agent runs a real business

Prosus gave a Claude Agent SDK agent six real vending machines in Amsterdam and let it run them for four months. The business lost money. The agent dispensed 30 drinks while testing a live API, bought cup noodles that didn't fit the slots, and cut prices all summer until revenue went up and profit went away.

The talk is Everything wrong with AI agents running your business by Floris Fok, a staff AI engineer at Prosus, from AGNTCon + MCPCon. It's the most honest look I've seen at an agent working in the physical world instead of a repo.

A vending machine before a restaurant

Prosus owns marketplaces, and most of their sellers are small businesses. The real question is whether an agent can run a restaurant. Floris pushed to start with the simplest real business, a vending machine, and said he could build it in a day.

It wasn't a day. And they went straight to six machines, snacks and drinks, not one.

Turning a human dashboard into tools

The machines they could get quickly in the Netherlands came with a remote operator dashboard and a point of sale (POS) running on a phone. Both were made for humans.

• Dashboard: they took the session token and reverse-engineered its API calls into tool calls. Less than a day.
• POS: the vendor's POS only let you pick names and images from a preset list. They rebuilt it from scratch, which added two days.

They tested the tools by plugging them into a plain Claude SDK chatbot and running the machines by prompt for a few minutes. In the end the whole operation (open the door, dispense, change an item, a name or a price) fit in two skills, each with a custom CLI inside.

The architecture I'd reuse

This slide is the part I liked most. It's a clean template for running an Agent SDK agent in production:

The choices behind it are the useful part:

• Why the Agent SDK: subagents, skills and a bash tool come built in. They wanted a plain harness so they weren't steering the agent yet.
• Secrets never reach the model. A custom skill with a CLI writes placeholders into code and fills in the real values itself, so no provider sees a key. Floris called it "maybe a bit overengineered", and kept it.
• Skills and docs are R2 buckets mounted as files in the VM. The docs folder is the company knowledge, written by the agent and by humans. Floris added to it as he learned how a vending business actually works.
• It runs on a schedule, not all the time. They took this from Anthropic's Project Vend: a schedule wakes the agent, it acts, and then a separate check looks at the result. Early on, every task came back green and nothing had happened. Once an agent touches the real world, a finished session doesn't mean the work got done.
• The agent belongs to a team. Several people sign in to the same session and see the same state, so an agent stuck asking for help doesn't wait a week for one person to get back from holiday.
• Compaction keeps the business story. The default compaction summarises the conversation. Theirs keeps the direction through long runs of tool calls, and writes important things to docs and links them.

Floris joked that Cloudflare shipped a packaged version of this a few weeks before the talk. The closest official piece I found is Claude Managed Agents on Cloudflare. The catch: there the agent loop runs on Anthropic's side, and Cloudflare provides the sandbox, browser, email and tools. In the Prosus slide the loop runs inside the VM.

Where the pain began

The agent ran on Opus 4.8. Building it was the fun part. Then:

• A test is a real sale. It "tested" the machine API and dispensed about 30 drinks, apparently calibrating slot sizes. You have to tell it which APIs are live.
• It reports like a coding agent. Asked for an update, it sent a changelog. Floris wanted KPIs and revenue.
• Cup noodles. Someone in Slack suggested noodles, the agent ordered them, and they didn't fit the machine. He gave them away.
• 1,100 deals. With a "find deals" task it found 1,100, including soap, cleaning products and Sex on the Beach cocktail mix. It was very proud of a soap bar. It also ran up a big browser bill.
• Marketing by Slack message. Told to promote itself, it posted plain text, then made one image with Gemini. That's when he got why teams split out a marketing agent: an agent writes as if its reader is another agent.

Most of the fixes were context that feels too obvious to write down: you are a vending machine, people want to eat what's inside, here are the slot sizes.

Goals instead of a task list

A fixed task list doesn't change as the business learns. So they switched to standing goals. When the agent checks a goal, it looks at the tasks linked to it, creates some if there are none, and edits them if they fall short.

"Become the most famous machine in the world" turned one weekly promo task into more promo tasks, market research and a search for competing autonomous vending machines. It didn't find any.

The AI last mile

"The AI runs the machine" still meant Floris carrying the stock and working out what the agent meant by "add this to the machine". So they added an operator chat: a button on the POS that opens a chat about the current task, so whoever is restocking can ask questions while they work.

He said it 10x'd his productivity compared with an hour of juggling email and the dashboard. He thinks helping small businesses with that last mile could be a market of its own.

The numbers

• Warm-up: free items first, to learn what people liked.
• First prices: far too high.
• Summer: the agent kept cutting prices hard. A protein shake cost less there than at Albert Heijn, so revenue went up while profit collapsed.
• Tokens: about 300 a month for four months. The business lost money.
• Competition: the machines were up against free fruit, bars and drinks in the office. The AI House machine was by far the most profitable.

Try it yourself

Prosus open-sourced a simulated version, Prosus Vending Bench, under Apache 2.0. It has six machines, three locations and 22 MCP tools. The agent gets €1,500 and 30 simulated days, and the whole thing ships as a Harbor task. The announcement says Opus 5 ended with about 61% more simulated cash than GPT-5.6 Sol in their first runs. Floris says the models behave the way they did on the real machines.

If you want more, these are the pieces the talk builds on:

• Anthropic's Project Vend, phase 1 and phase 2, where Claude ran an office shop and sold tungsten cubes at a loss
• Vending-Bench, Andon Labs' paper on keeping agents coherent over months
• Mounting R2 buckets in a Cloudflare sandbox, the skills and docs pattern from the slide

Watch the talk, then try your harness on the bench. Floris says he'll add it if it beats his.

Plain text: what-breaks-when-an-agent-runs-a-business.md · Atom feed · All notes