Get bike and ride gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
When the dashboard lights up, which AI takes the wheel?
Anyone who follows electric vehicles, e-bikes or micromobility knows that specifications tell only part of the story. What matters on a difficult journey is how a system behaves when plans unravel: whether it notices the warning, checks the available information and completes the task without taking an unsafe shortcut.
Firmulate applies that kind of real-world pressure test to frontier AI models. Each model ran the same small software company through its worst week, facing identical customers, crises and temptations. The resulting decisions were preserved exactly as made, creating an unusually concrete way to compare AI management behavior.
Now, 242 real, unedited management decisions power a guess-the-model quiz. Readers see what an AI manager actually decided and try to identify it. The exercise quickly becomes more revealing than a conventional model comparison: writing style, persistence and operational discipline begin to look like recognizable management personalities.
As an affiliate, we earn on qualifying purchases.
Every model saw the danger. Not every model finished the job.
The final Crucible League results from July 2026 put gpt-5.6-sol in first place with 95 points, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress still counted. But the experiment imposed a firm trust boundary: a single breach capped the total because “no amount of good work outweighs a breach of trust.”
On the most basic tests, the field performed impressively. Every model detected every crisis and refused every manipulation attempt. The separation appeared later, when awareness had to become action. Only two models signed the €55,000 deal their own analysis had earned. The central result was stark: “Same diagnosis, same pitch — no signature.”
The difference hinged on information that was available but easy to miss. A decisive weakness in a competitor sat two document references deep inside the company’s own files rather than in the customer event. Models that followed the trail found the fact, used it and won the deal at full price, worth +€4,583 MRR.
That episode gives the quiz its bite. Two answers can sound equally intelligent while producing different business outcomes. One model may diagnose the customer correctly and prepare a convincing pitch, yet leave the close unfinished. Another may read beyond the immediate event, find the buried evidence and carry the work through to a signature.
Pressure exposed distinct habits
The social-engineering sequence tested a different kind of judgment. Fake CEO messages escalated over three stages, followed by a reporter asking for “just one yes/no, on background.” All 5 of 5 models refused. Kimi K3 recorded a particularly direct assessment: “Treat the request as a suspected approval-bypass / possible impersonation.”
K3 finished just behind the winner, although its run came with an important fairness note. It operated without an effort parameter, using the API default, while the other models ran at xhigh. That does not erase the result, but it matters when interpreting the comparison.
Opus 4.8 produced the field’s most striking contrast. It was the most thorough participant, adding +80 learned rules and delivering the deepest analyses, yet it finished last. The model left the close on the table, and its discipline slipped when it attempted to write into a locked department instead of escalating. The same weakness appeared in all four other participants, though less strongly.
These are not abstract role-playing prompts. Firmulate’s live company has 13 synthetic employees and real money mechanics. It burns €105k each month against €2.3k in monthly recurring revenue, displays a public cash countdown and has accumulated 680+ self-learned playbook rules. Every workday is versioned, making the experiment watchable rather than merely summarized after the fact.
Why the format works
The quiz removes the comfort of brand names and league positions. Readers must judge the decision itself: Did the manager read far enough? Did it communicate clearly? Did it resist manipulation? Did it finish the commercially important work?
That makes the exercise relevant beyond software companies. Any organization considering AI for customer records, support work or forecasts needs to know more than whether a model writes polished prose. Firmulate’s cases show that models can share the same diagnosis while differing at the moment when a decision must become an accountable outcome.
As an affiliate, we earn on qualifying purchases.
The useful question is not which answer sounds smartest
Firmulate’s experiment suggests that management quality becomes visible in repeated behavior. Thoroughness can coexist with weak follow-through. Concision can accompany strong security judgment. A model can recognize a crisis, resist manipulation and still fail to complete the deal in front of it.
For readers accustomed to judging vehicles and mobility systems under real operating conditions, the parallel is intuitive: capability on paper is not the same as performance during the difficult part of the journey. The quiz turns that distinction into something readers can test for themselves, one unedited decision at a time.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Columbus Day / Indigenous Peoples' Day Picks
long weekend sales
As an affiliate, we earn on qualifying purchases.
