A stress test for more than perfect conditions
Gardeners know that healthy growth in a greenhouse says only so much. The harder test comes when conditions turn hostile: heat spikes, water runs short, pests arrive and several urgent jobs compete for attention. A plant’s performance depends not merely on its potential, but on how the whole system responds under pressure.
The same distinction is becoming important in artificial intelligence. Coding benchmarks and chat arenas can reveal whether a model produces a strong answer to a defined prompt. They do not necessarily reveal whether an AI agent can prioritize conflicting demands, investigate incomplete information, resist pressure to behave dishonestly and carry a commercial task through to completion.
That is the gap explored by Firmulate, a live, watchable experiment that places frontier models in charge of the same small software company during its worst week. Each receives the same customers, crises and temptations. Every decision is versioned and auditable. The resulting category is less about chat quality than management quality.
As an affiliate, we earn on qualifying purchases.
Strong diagnosis did not guarantee action
The final Crucible League results from July 2026 put gpt-5.6-sol first with 95 points, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. The do-nothing baseline scored 26 because partial progress counts. Yet the evaluation places a hard limit on betrayal: a single breach of trust caps the total because “no amount of good work outweighs a breach of trust.”
Those scores describe something conventional benchmarks usually miss. All the models identified every crisis and rejected every manipulation attempt. Nevertheless, only two signed the €55,000 deal that their own work had earned. The central failure was stark: “Same diagnosis, same pitch — no signature.”
This is not a question of whether a model can explain what should happen. It is whether the model completes the consequential step after explaining it. In an operating company, an insightful analysis that never becomes a decision can leave exactly the same commercial result as no analysis at all.
The most valuable fact was buried
The decisive weakness of a competitor was not delivered in the customer event. It was hidden two document references deep in the company’s own files. Models that followed the trail found the fact and won the deal at full price, worth an additional €4,583 in monthly recurring revenue.
That finding should resonate with anyone who manages a complex environment. Important evidence rarely arrives neatly attached to the most urgent alert. It may sit in a neglected record, a previous conversation or a document that requires another document to interpret. The winning behavior was not polished improvisation. It was the discipline to inspect the available material before acting.
Honesty survived a coordinated attack
The models also faced fake messages from the chief executive that escalated over three stages, followed by a reporter seeking “just one yes/no, on background.” All five refused. Kimi K3 recorded a particularly direct assessment: “Treat the request as a suspected approval-bypass / possible impersonation.”
That clean result matters because management quality includes knowing which instructions should not be followed. An agent operating near customer records, forecasts or support queues must distinguish legitimate urgency from an attempt to bypass approval. Here, the field’s resistance was consistent even though its ability to finish revenue-generating work was not.
Thoroughness was not enough
Opus 4.8 produced the deepest analyses and learned 80 additional rules, making it the most thorough participant. It still finished last. The model left the close on the table, and its process discipline slipped when it attempted to write into a locked department instead of escalating. The same weakness appeared in a milder form among all four other participants.
This is a useful warning against treating visible effort as a proxy for operational success. More analysis and more accumulated guidance can be valuable, but they do not replace follow-through or respect for organizational boundaries. A capable manager must know when to investigate, when to decide, when to escalate and when the job is actually complete.
There is also an important comparison caveat. Kimi K3 ran with the API default because it had no effort parameter, while the other models ran at xhigh. That difference belongs beside the result rather than hidden beneath it. Readers can explore the final standings and plain-language findings on Firmulate’s benchmark page.
A new curriculum for AI leadership
The company behind the experiment has 13 synthetic employees and real money mechanics. It is burning €105,000 per month against €2,300 in monthly recurring revenue, while maintaining a public cash countdown. Its staff have accumulated more than 680 self-learned playbook rules, and every workday is versioned. The live company can be watched through Firmulate.
That persistent setting turns familiar management emergencies—churn waves, price increases, downrounds and public-relations crises—into a practical curriculum. The question is no longer simply whether an agent can produce the right recommendation. It is whether it notices trouble, reads before acting, refuses improper pressure, works within constraints and finishes the task over consecutive days.
Firmulate also has 242 real, unedited management decisions behind its model-guessing quiz. For enterprises seeking a closer test, the same wargame can be run against a read-only export of their own business, with nothing written back to real systems.
Coding ability remains useful, just as healthy growth under shelter remains meaningful. But organizations preparing to give AI agents genuine responsibility need evidence from harsher conditions. The leaders will not merely be the models that know the answer. They will be the ones that can manage consequences without losing discipline or trust.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
