AI Benchmarks & Leaderboard — 2026-09-06

New Model Releases & Updates

1 sources

zdnet.com

zdnet.com

Claude Fable 5.1 by Anthropic

• Type: Closed-source
• Key benchmarks: Artificial Analysis Intelligence Index: 57 (Leading among 52/53 reasoning models)
• vs. Previous best: Represents a significant leap in reasoning capabilities, positioning it at the very top of the current frontier for complex logic tasks.
• What's notable: The model is specifically tuned for "Adaptive Reasoning" with a "Max Effort" setting, optimizing for high-stakes problem-solving scenarios.

OpenAI Astra by OpenAI

• Type: Closed-source (Frontier)
• Key benchmarks: Not explicitly detailed in search results, but claimed to handle computer and browser use tasks with "unmatched speed, accuracy, and safety."
• vs. Previous best: Positioned as a "new frontier" specifically for agentic tasks involving computer interaction, distinguishing it from general-purpose chat models like GPT-5.5.
• What's notable: The launch is described as "controversial," likely due to its capabilities in autonomous browser/computer control.

Gemini 3.7 Flash by Google

• Type: Closed-source (Flash tier)
• Key benchmarks: Improved Intelligence Index by 4 points over Gemini 3.6 Flash.
• vs. Previous best: Reaches the Intelligence vs. Speed frontier, offering a significant upgrade over its immediate predecessor while maintaining high throughput.
• What's notable: Focuses on balancing the "Intelligence vs. Speed" tradeoff, making it suitable for latency-sensitive applications requiring higher reasoning than standard flash models.

Qwen3.8 2.4T A95B by Alibaba

• Type: Open-weights (Mixture of Experts)
• Key benchmarks: Ranked as the top open-weights model on the Artificial Analysis leaderboard.
• vs. Previous best: Sets a new bar for open-source intelligence, leveraging a massive 2.4 Trillion parameter architecture with an Active 95B parameter configuration (A95B).
• What's notable: Demonstrates that sparse MoE architectures can achieve frontier-level intelligence while keeping inference costs manageable via active parameter sparsity.

Leaderboard Snapshot

Frontier Models (Closed-Source)

Open-Source Leaders

Benchmark Deep Dive: The Intelligence Index Hierarchy

The Artificial Analysis Intelligence Index continues to serve as a primary metric for comparing frontier capabilities. As of early September 2026, Claude Opus 4.8 holds the absolute top position with a score of 61, driven by its "Adaptive Reasoning, Max Effort" configuration. This suggests that when compute is not constrained, Anthropic's latest Opus model offers the deepest capacity for complex, multi-step logical deduction.

However, a distinct tier has emerged below the absolute ceiling. GPT-5.5 follows closely at 60 (xhigh effort), while Claude Fable 5.1 and Gemini 3.1 Pro Preview sit at 57. The gap between the top 1 and the next tier (57-60) indicates diminishing returns in raw intelligence scores, or perhaps saturation in the specific benchmarks used to calculate the index. The differentiation now lies heavily in how these models achieve these scores—specifically through "effort" settings (e.g., xhigh vs. high) which trade off latency and cost for marginal gains in accuracy.

For practitioners, the data reveals a bifurcation: Mercury 2 dominates the speed metric at nearly 900 tokens per second, vastly outperforming other models like Step 3.7 Flash (~407 t/s). This highlights that while Opus 4.8 leads in intelligence, it is likely not the choice for real-time, low-latency applications unless paired with speculative decoding or similar acceleration techniques. Meanwhile, the entry of OpenAI Astra signals a shift away from pure "chatbot" benchmarks toward agentic performance, where metrics like browser navigation accuracy and task completion rates will become the new frontier for evaluation.

Analysis & Trends

• State of the art: Anthropic currently holds the top two spots in the intelligence index (Opus 4.8 and Fable 5.1), suggesting a slight lead in pure reasoning over OpenAI's GPT-5.5 and Google's Gemini 3.1 Pro. However, OpenAI's introduction of Astra suggests they are pivoting toward specialized agentic capabilities rather than just general chat intelligence.
• Open vs. Closed gap: The gap remains significant but narrowing in terms of architecture efficiency. Qwen3.8 leading the open-weights category with a 2.4T MoE architecture proves that massive scale (even if sparse) is still required to compete with dense closed-source models. Open models like DeepSeek V4 and Kimi K3 are becoming viable alternatives for coding and reasoning tasks where data privacy or cost is paramount.
• Cost-performance: Qwen3.5 0.8B is noted as the most affordable option at $0.01 per 1M tokens (blended), indicating that small language models (SLMs) are becoming increasingly capable of handling basic tasks at negligible cost. Conversely, "Max Effort" modes on frontier models likely carry a premium price tag due to increased token generation during internal reasoning steps.
• Emerging patterns: The "Effort" parameter is becoming a standard toggle. Models are no longer just "smart" or "dumb"; they have dynamic compute allocation (e.g., Adaptive Reasoning). Users must now tune "effort" levels (low, medium, high, max) to balance the Intelligence vs. Cost/Speed tradeoff.

What to Watch Next

• OpenAI Astra's Safety & Agentic Benchmarks: Given the "controversial" nature of Astra's launch, independent evaluations of its safety guardrails and actual performance in autonomous web tasks will be critical.
• Gemini 3.7 Flash Adoption: Developers will test if the 4-point intelligence boost over 3.6 Flash justifies any potential cost increases or if it becomes the new default for high-volume, medium-complexity tasks.
• Qwen3.8 Deployment Challenges: While Qwen3.8 leads in open weights, the hardware requirements for 2.4T parameters (even with A95B sparsity) will drive interest in quantization techniques and efficient serving frameworks.