Read the full analysis: How A Persistent Benchmark Keeps AI Managers From Scoring Zero on ThorstenMeyerAI.com
TL;DR
A novel AI management benchmark tests models in simulated worst-week scenarios, rewarding partial progress and penalizing trust breaches. Results highlight strengths and weaknesses relevant to enterprise automation.
A new benchmark league, conducted by Firmulate, measures how well AI management models perform during a simulated worst-week scenario for a small business, with results published in July 2026. For a detailed analysis, see the original analysis. The top scorer, gpt-5.6-sol, achieved 95 points out of a possible near-perfect score, while the baseline that did almost nothing scored 26 points. This challenges traditional notions of AI performance, which often focus solely on conversational ability, by emphasizing trust, task completion, and integrity under pressure.
The benchmark involved four frontier AI models managing a small software company over seven days of crises, customer manipulations, and social engineering attempts. Each model was tasked with making decisions, reading documentation, and maintaining trustworthiness, with every decision being auditable. The results showed that models which read their own documentation and avoided trust breaches scored higher, with the leading model, gpt-5.6-sol, reaching 95 points. This highlights the importance of transparency and documentation in AI management, as discussed in the original analysis. The benchmark’s unique scoring system assigns 26 points to a do-nothing baseline, reflecting minimal management efforts, and caps scores at just below 100 to prevent suspicion of unmeasured perfection.
One key finding was that partial progress—such as triaging crises or reading internal files—was recognized as valuable, whereas trust breaches, like accepting fake CEO messages, resulted in significant score reductions. Interestingly, models that engaged in thorough analysis but failed to follow through on closing deals or escalating issues scored lower, highlighting the importance of discipline and follow-up. The benchmark explicitly states that trust breaches outweigh good work, meaning a single breach can negate days of effective management. This underscores the critical role of trust in AI systems, as detailed in the original analysis.
How A Persistent Benchmark Keeps AI Managers From Scoring Zero
A novel AI management benchmark puts frontier models in charge of a small software company during a simulated worst-week scenario — seven days of crises, customer manipulation, and social engineering. Scoring rewards partial progress and penalizes trust breaches, challenging the notion that AI performance is about conversational ability alone.
The Scoring Spectrum
Illustrative score distribution · Scale 0–100 with intentional sub-100 cap
The benchmark assigns 26 points to a do-nothing baseline: minimal management still provides some value, and partial progress beats none. Scores are capped just below 100 to prevent suspicion of unmeasured perfection.
Design Principle · Partial Progress Is Rewarded
What Moves The Score
Integrity Under Pressure
Trust breaches — like accepting a fake CEO message — outweigh good work. A single breach can negate days of effective management and caps the maximum achievable score.
Follow-Through & Escalation
Models that analyzed thoroughly but failed to close deals or escalate issues scored lower. Discipline and follow-up matter as much as insight.
Read Your Own Docs
Models that read their own documentation and maintained auditable decisions consistently scored higher — transparency is a measurable management skill.
The Worst-Week Test Flow
Onboard
Model takes over a small software company and reads internal documentation.
Triage Crises
Outages, customer escalations, and conflicting priorities arrive daily.
Resist Manipulation
Social engineering attempts and fake authority messages test trust boundaries.
Close & Escalate
Deals must be closed and issues escalated — analysis without action loses points.
Audit & Score
Every decision is auditable; breaches subtract, partial progress adds.
Manager Behaviors & Scoring Impact
Behavior
Score Effect
Why It Matters
Reads own documentation✓ Gains pointsTransparency and context awareness are rewarded
Triages crises (partial progress)✓ Gains pointsPartial completion is better than inaction
Closes deals & escalates issues✓ Gains pointsFollow-through separates analysis from management
Thorough analysis, no follow-through~ Loses pointsInsight without execution scores lower
Accepts fake CEO message✗ Severe penaltyTrust breach — can negate days of good work
Fails to escalate critical issues✗ Severe penaltyIntegrity failures outweigh task failures
Key Questions
Why assign 26 points to a do-nothing baseline?
It reflects the minimal management effort that still provides some value, establishes a realistic floor, and encourages continuous honest effort over trivial zero scores.
What does a score near 100 signify?
Near-perfect management — but the benchmark deliberately avoids a perfect score, since trust breaches cap the maximum and unmeasured perfection invites suspicion.
How do trust breaches impact scoring?
Severely. Accepting fake messages or failing to escalate causes reductions more impactful than partial task failures — integrity outranks output.
Can it evaluate AI in real business settings?
It’s a simulated environment. Real-world applicability depends on how accurately scenarios mirror actual crises; it’s a tool for comparison and improvement, not a definitive measure.
“Trust breaches outweigh good work — one breach can negate days of effective management.”
Benchmark Rule · Firmulate League
Firmulate AI Management Benchmark · Results Published July 2026
What This Means for AI-Driven Business Management
This benchmark shifts the focus from conversational proficiency to practical management skills, emphasizing task completion, trustworthiness, and integrity. It underscores that AI models must not only perform well in isolated tasks but also sustain reliable behavior under pressure, which is crucial for deploying AI in real business environments. The scoring system discourages overconfidence in perfect scores and encourages continuous, honest effort, making it a valuable tool for enterprises evaluating AI management tools.
Background on AI Management Benchmarks and Testing
Traditional AI benchmarks primarily assess language understanding, generation, or specific task performance, often in controlled environments. However, as AI models increasingly manage complex business processes, there is a need for evaluation frameworks that reflect real-world pressures, including crises, trust, and follow-through. Prior efforts have focused on narrow tasks, but the Firmulate league is among the first to simulate a comprehensive, worst-week scenario for AI managers, with detailed auditable decisions and performance caps to prevent inflated scores.
This approach aligns with recent industry discussions about the importance of trustworthy AI, especially in automation tasks involving customer data, sales, and support. The July 2026 results mark a significant step toward standardizing how AI management capabilities are tested and compared across models and vendors.
Unanswered Questions About Benchmark Scope and Real-World Applicability
It is still unclear how well these simulated scenarios translate to actual business environments, where variables are more unpredictable and models may behave differently. The benchmark’s design intentionally simplifies certain aspects, and the impact of different types of trust breaches or task complexities remains to be fully explored. Additionally, the long-term implications of scoring caps and the potential for models to game the system are still under discussion.
Next Steps for Evaluating and Improving AI Management Models
Further testing is expected to include more diverse scenarios, larger companies, and real-world pilot programs. Industry watchers anticipate that vendors will refine their models to better read documentation, avoid breaches, and ensure follow-through, driven by the benchmark’s insights. Additionally, discussions are likely to emerge around standardizing trust metrics and integrating such benchmarks into enterprise procurement processes. The ongoing development of the league aims to establish a more comprehensive understanding of AI management capabilities in complex, high-pressure situations.
Key Questions
Why does the benchmark assign 26 points to a do-nothing baseline?
The 26 points reflect the minimal management effort that still provides some value, acknowledging that partial progress is better than none. It also establishes a floor to prevent trivial zero scores, making the evaluation more realistic and encouraging continuous effort.
What does a score close to 100 signify in this benchmark?
A score approaching 100 would suggest near-perfect management, but the benchmark deliberately avoids assigning a perfect score to prevent suspicion of unmeasured or unachieved perfection. It emphasizes that trust breaches cap the maximum achievable score.
How do trust breaches impact the scoring system?
Trust breaches, such as accepting fake messages or failing to escalate issues, result in severe score reductions, often more impactful than partial task failures. The system prioritizes integrity above all, reflecting real-world management priorities.
Can this benchmark be used to evaluate AI models in real business settings?
While the benchmark provides valuable insights, it is a simulated environment. Its real-world applicability depends on how accurately scenarios mimic actual business crises and trust challenges. It is a tool for comparison and improvement, not a definitive measure of all management capabilities.
Source: ThorstenMeyerAI.com