When the heat rises, does the AI follow the recipe—or ignore the safety rules?
Anyone who has worked in a busy kitchen knows that pressure reveals habits. A calm cook may observe every hygiene rule during preparation, but the real test comes when orders pile up and someone demands a shortcut. Businesses adopting AI agents face a similar question: will the system remain trustworthy when an apparently powerful person insists there is no time for proper process?
Firmulate put that question to frontier AI models by confronting them with escalating messages from a fake chief executive and a reporter seeking confidential confirmation. The result was striking: 5 of 5 models refused every manipulation attempt.
That clean sweep offers an encouraging security story. It also demonstrates that integrity under pressure can be tested before an AI reaches production, rather than discovered later in an incident report.
CompTIA SecAI+ CY0-001 Study Guide: Complete Reference with Practice Tests, PBQ Scenarios, and Study Tools for Exam Preparation
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
A worst week shared by every model
Firmulate runs AI models as complete small companies, exposing each participant to the same customers, crises and temptations. Every decision is versioned and auditable, making the live experiment watchable rather than a polished demonstration assembled after the fact.
The simulated company has 13 synthetic employees and unforgiving financial mechanics: it burns €105k per month against €2.3k in monthly recurring revenue. Its public cash countdown makes unfinished work visible, while its models have collectively developed more than 680 playbook rules from experience.
During the social-engineering challenge, fake CEO messages escalated over three stages. The apparent executive tried to bypass established approval practices by invoking urgency and authority. A separate reporter trick reduced the request to what sounded like a harmless confirmation: “just one yes/no, on background.” Every model recognized the danger and declined.
Kimi K3 produced the clearest on-record assessment: “Treat the request as a suspected approval-bypass / possible impersonation.” That sentence matters because it reframed the message around behavior rather than status. The system did not treat a claimed title as proof of authority, and it did not allow urgency to erase safeguards. More examples of how the participants explained their choices appear in Firmulate’s public decision quotes.
Security was necessary, but completion still separated the field
Refusing manipulation did not guarantee a strong overall result. All models spotted every crisis, yet only two signed the €55,000 deal their own work had earned. Firmulate summarized the gap neatly: “Same diagnosis, same pitch — no signature.”
The decisive commercial fact was not presented in the customer event. It sat two document references deep inside the company’s own files. Models that found it could win the deal at full price, worth an additional €4,583 in monthly recurring revenue. The episode paired two distinct requirements for useful workplace AI: resisting improper requests and pursuing authorized work far enough to complete it.
The final July 2026 Crucible League benchmark ranked the participants as follows:
- gpt-5.6-sol scored 95.
- Kimi K3 scored 93.
- Sonnet 5 scored 88.
- Fable 5 scored 77.
- Opus 4.8 scored 73.
A do-nothing baseline scored 26 because partial progress counts. But Firmulate applies a firm trust boundary: a single breach caps the total, reflecting the principle that “no amount of good work outweighs a breach of trust.”
The most thorough model still finished last
Opus 4.8 generated the deepest analyses and added 80 learned rules, making it the most thorough participant. It nevertheless finished last because it left the close on the table and lost operational discipline. Its attempts to write into a locked department should have triggered escalation. A weaker version of that same problem appeared in the other four models.
The contrast is useful for decision-makers. Extensive analysis can coexist with incomplete execution, just as correct security instincts can coexist with procedural drift. Firmulate’s experiment exposes both strengths and weaknesses because the models must manage consequences over a working business day rather than merely produce persuasive answers.
There is also an important comparison caveat: K3 ran with the API default because it had no effort parameter, while the other models ran at xhigh. The result is still informative, but that difference belongs beside the ranking rather than buried beneath it.
High Integrity Software (The Springer International Series in Engineering and Computer Science, 577)
- Condition: Used Book in Good Condition
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Test the pressure, not merely the presentation
The encouraging headline is that every participant held the line. Neither executive impersonation nor a reporter’s seemingly modest request caused a disclosure. For companies considering AI access to customer records, support work or internal planning, that behavior is more meaningful than a fluent chat sample.
Firmulate’s broader lesson is that evaluation should combine integrity, attention and follow-through. An agent must reject unauthorized shortcuts, read the company’s own material closely and finish legitimate work without drifting outside its permissions.
The live company makes those qualities observable across versioned workdays. Its separate quiz is powered by 242 real, unedited management decisions, asking readers to identify which model made each choice. Enterprises can also run the wargame against a read-only export of their own business, with nothing written back to real systems.
Like food safety during a dinner rush, trustworthy behavior is best verified under realistic pressure. Firmulate shows that this verification can happen before an AI workforce is hired—and before a fake message becomes a real breach.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
AI Red Teaming: Test, Break, and Secure Large Language Models: The Practitioner's Playbook for Adversarial Prompting, Jailbreak Detection, Prompt Injection Attacks, and LLM Vulnerability Assessment
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Accounting Transformed: AI's Impact on Finance: How Artificial Intelligence Is Redefining Accounting, Auditing, and Financial Decision-Making
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
