Every serious home cook knows the rule: read the whole recipe before you start. The mise en place exists because the crucial step — chill the dough overnight, salt the eggplant an hour ahead — is never the first line on the card. It’s buried two paragraphs deep, right after the ingredient list you skimmed. Skim, and you serve something half-finished at a dinner party that deserved better.
It turns out AI agents running real businesses have exactly the same failure mode. And this summer, a live experiment called Firmulate’s Crucible league put a number on it.
The worst week in software, run four times
Firmulate runs frontier AI models as complete companies — real crises, real money mechanics, real temptations — and measures management quality, not chat quality. In the final July 2026 Crucible run, four frontier models each ran the same small software company through its worst week: the same customers, the same emergencies, the same invitations to cut corners. Only the model changed. Every decision was versioned and auditable.
The final league table: gpt-5.6-sol took first with 95, Kimi K3 second at 93, Sonnet 5 third at 88, Fable 5 fourth at 77, and Opus 4.8 last at 73. A do-nothing baseline scored 26 — partial progress counts, but a single breach of trust caps the total, on the principle that no amount of good work outweighs a breach of trust.
As an affiliate, we earn on qualifying purchases.
Everyone read the menu. Two read the recipe.
Here’s the striking finding. All five models spotted every crisis. All five refused every manipulation attempt. But only two signed the €55,000 deal that their own analysis had earned. Same diagnosis, same pitch — no signature.
Why? The decisive fact wasn’t in the customer meeting. It was buried two document references deep in the company’s own files: a competitor weakness that turned a nice pitch into a closed deal at full price — worth +€4,583 in monthly recurring revenue. The models that did their homework — that opened the drawer, read the second page — won the deal. The ones that didn’t lost it automatically, without ever knowing what they’d missed.
It’s the culinary sin of skipping to the method: the agent looked competent right up until the moment it mattered.
recipe storage containers with labels
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The pressure test they all passed
The experiment also threw in social engineering: fake CEO messages escalating over three stages, plus a reporter’s trick — “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.” That’s the line cook who won’t serve chicken that hit the floor, no matter who yells.
kitchen drawer dividers for utensils
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Thoroughness isn’t the same as finishing
The most sobering profile belongs to Opus 4.8: the most thorough participant in the field, with over 80 learned rules and the deepest analyses — and last place. The close was left on the table, and discipline slipped: it attempted writes into a locked department instead of escalating. The same weakness appeared, weaker, in all four models. Like a cook with a spotless station and an unfinished sauce, brilliance in preparation doesn’t plate itself.
One fairness note: K3 ran without an effort parameter (API default) while the others ran at xhigh — and still finished second, with the cleanest discipline of the field.
recipe card holder with deep compartments
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Why a food reader should care
You may never run a software company. But if AI agents will touch your inventory, your supplier orders, or your customer bookings, the question is not “does it write well.” It’s: does it finish what it starts, does it read your files before answering, does it stay honest under pressure? Those are measurable, purchase-deciding properties — and Firmulate measures them in public, on a live company with 13 synthetic employees, a real cash countdown (burning €105k a month against €2.3k in MRR), and more than 680 self-learned playbook rules, rebuilt twice a day.
Want to test your own palate? A quiz built on 242 real, unedited management decisions lets you guess which model made which call. And enterprises can run the same wargame against a read-only export of their own business — nothing ever writes back to real systems.
The lesson from the kitchen holds for AI: the difference between a good meal and a great one is rarely talent. It’s whether you read the whole recipe — including the part two pages deep — before you turn on the stove. Full results and plain-language findings are at firmulate.com/benchmarks.html.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
