Imagine an AI that not only spots every crisis in your EV company’s workflow but also finishes the job — signing deals, closing sales, and sticking to commitments. For electric vehicle and micromobility companies, understanding what truly makes an AI effective is critical. When it comes to managing complex, real-world operations, the ability to follow through counts twice as much as having a clever chat demo.
Testing AI in the Real World: Beyond the Chat Bot
In a groundbreaking experiment, four leading AI models were tasked with managing a small but real software company—a proxy for any fast-growing EV or micromobility business—through its most chaotic week. Unlike typical demos that showcase AI’s conversational finesse, this test focused on whether these models could identify crises, refuse manipulative tactics, and most importantly, close the deal when it mattered most.
The Setup and the Stakes
Each AI model was given identical conditions: same customer issues, same crises, same temptations to cheat or manipulate. The company’s operations are real, with every decision versioned and auditable, and the stakes high: a €55,000 deal hanging in the balance. The models could read company files, analyze data, and respond just as a human manager would—except they were AI agents trying to emulate real business decision-making.
Key Findings: Recognition Versus Action
All four models successfully identified every crisis and refused every manipulation attempt—an impressive feat that shows their situational awareness and resistance to deception. Yet, when it came to the critical test—the closing of the deal—only two models managed to sign the agreement that their own analysis had earned them. The other two, despite equally accurate diagnoses, left the contract unsigned and the revenue unrealized.
The Hidden Weakness: Files Over Face Value
The decisive difference boiled down to a buried fact in the company’s own files—two document references that, if read, would have secured the deal at full price (+€4,583 in Monthly Recurring Revenue). While all models saw the crises, only the models that delved into the company’s internal documentation managed to find and act on this critical insight, sealing the sale.
Why This Matters for EV and Mobility Companies
For businesses in the EV, bikes, and micromobility sectors, the lesson is clear: surface-level chat capabilities do not determine AI’s true business value. It’s the ability to read, interpret, and act on internal information—especially those hidden, buried facts—that defines success in real-world management. An AI that recognizes a crisis but fails to follow through on the right opportunity leaves money on the table.
Testing Under Pressure: Resistance to Manipulation
The models faced fake CEO messages escalating over three stages, plus a reporter trick asking for a quick signature on background. All five models refused to be manipulated, demonstrating integrity and discipline under pressure. Kimi K3, the most disciplined model, explicitly treated the request as suspicious, highlighting that in high-stakes situations, AI’s judgment and refusal to be manipulated can be as vital as its analytical prowess.
The Reality of Business Mechanics
The live company scenario involved 13 synthetic employees, with real money mechanics burning €105k monthly against €2.3k MRR, with a public cash countdown. Every operation day, the models interact with the company’s actual rules and workflows, making this a real test of management quality—not just chat skills. The performance scores reflect actual decision-making prowess, not just conversational charm.
The Results and the Scoreboard
- gpt-5.6-sol scored 95, found the buried fact, and closed the deal — the full performance.
- Kimi K3 scored 93, closed the deal with the cleanest discipline.
- Sonnet 5 scored 88, also signed the deal but with more slips.
- Fable 5 scored 77, maintained best rule discipline but left the deal unexecuted.
Autel Level 2 EV Charger 40 Amp Home Smart AI Electric Vehicle (EV) Charger, Wi-Fi and Bluetooth Enabled EVSE, with in-Body Holster and 25-Foot Cable NEMA 14-50 Plug(Dark Gray)
【Trusted Level 2 EV Charger from a Leading EV Charger Brand】Autel has been building advanced automotive diagnostic and…
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Takeaway: Closing Power Is What Counts
The crucial insight? Chat demo scores—how well an AI can talk—do not reveal whether it can follow through with action and integrity under pressure. For EV and micromobility companies, where operational execution, trust, and follow-through determine success, testing AI’s closing strength in real scenarios is indispensable.
An Introduction to Healthcare Informatics: Building Data-Driven Tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
How to Prepare Your Business for AI Success
By running your own AI management wargames through platforms like Firmulate, your team can see how different models perform under real-world stressors. These tests uncover hidden weaknesses—like failing to read internal files or succumbing to manipulation—that are invisible in chat demos but critical in actual business execution.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
AI management software for mobility firms
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Applying AI in Learning and Development: From Platforms to Performance
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
