AI Benchmarks & Leaderboard — 2026-11-03

New Model Releases & Updates

Mistral Large 4 by Mistral AI

• Type: Open-source (weights released), trillion-parameter system
• Key benchmarks: Top open-weight performance in agentic coding and cybersecurity AI benchmarks
• vs. Previous best: Claimed to be the "best open model built in the West," challenging closed models for enterprise control
• What's notable: Targets organizations concerned about losing access to closed AI, offering control over cyber defenses

  

Gemini 4 Carbon by Google

• Type: Closed-source, frontier model
• Key benchmarks: Ranked #1 on Artificial Analysis Intelligence Index with a score of 59 (High setting)
• vs. Previous best: Outperforms Claude Opus 5.5 and GPT-6 Astra on the new v5 index
• What's notable: Part of Google's October update which also included Gemini 4 Argon and Genie 4

  

GPT-6.1 Terra by OpenAI

• Type: Closed-source
• Key benchmarks: Included in recent benchmark comparisons; specific v5 index scores not explicitly detailed in available snippets but ranked among top models like GPT-6 Astra (Score 57)
• vs. Previous best: Part of the GPT-6 series evolution; noted for improved computer-use capabilities and lower pricing compared to previous flagships
• What's notable: Launched with 50% lower pricing and improved computer-use capabilities

Leaderboard Snapshot

Frontier Models (Closed-Source)

Note: Scores based on Artificial Analysis Intelligence Index v5.

Open-Source Leaders

Note: Specific aggregate scores for open models vary by benchmark; Kimi K3 recently fell from #4 on a major benchmark following updates.

Benchmark Deep Dive

The recent overhaul of the Artificial Analysis Intelligence Index to version 5 has had a profound impact on how model performance is perceived. By shifting 60 percent of the composite score to private test sets, the new methodology has widened the gap between proprietary and open-weight models from 3 points to 12 points. This change addresses concerns about data contamination in public benchmarks, forcing models to demonstrate generalization rather than memorization.

This shift has particularly affected Chinese contenders. For instance, Kimi K3.1 Pro, previously ranked fourth globally, saw a significant drop in standing after the benchmark update. This suggests that while open-weight models excel on public leaderboards, they may struggle more with unseen, private evaluations compared to their closed-source counterparts which have larger, more diverse training data pipelines.

For practitioners, this means that public leaderboard rankings are becoming less predictive of real-world performance on novel tasks. The "v5" methodology emphasizes robust reasoning and adaptability over raw score accumulation on known datasets. Consequently, enterprises choosing between open and closed models must now weigh the "privacy premium" of closed models against the control and customization benefits of open weights, especially as the performance gap in rigorous testing widens.

Analysis & Trends

• State of the art: Google's Gemini 4 Carbon currently leads the closed-source intelligence index, followed closely by Anthropic's Claude Opus 5.5. In the open-weight space, Mistral Large 4 is positioning itself as the leader for specialized tasks like cybersecurity and agentic coding.
• Open vs. Closed gap: The gap is widening on rigorous private benchmarks (12-point difference), despite open models often appearing competitive on public tests. This highlights the importance of evaluation methodology.
• Cost-performance: There is a notable trend toward lower pricing for high-end models, with GPT-6.1 Terra launching at 50% lower pricing than previous flagships. Open models like DeepSeek V4.1 Flash continue to offer extreme cost efficiency, sometimes approaching $0.00 per task in certain metrics.
• Emerging patterns: Agentic capabilities are becoming a primary differentiator. Models are being evaluated not just on static Q&A but on their ability to perform multi-step tasks, code execution, and security defense, as seen in Mistral's marketing of Large 4.

What to Watch Next

• Impact of Private Test Sets: Monitor how other benchmarking platforms adopt private test sets similar to Artificial Analysis v5 to prevent overfitting and provide a clearer view of true generalization.
• Open-Weight Cybersecurity Adoption: Observe whether enterprises actually adopt open-weight models like Mistral Large 4 for sensitive cybersecurity applications, validating the claim that open weights offer superior control over closed models.
• Next Generation Chinese Models: Watch for responses from Chinese labs (e.g., Moonshot, DeepSeek) to the recent benchmark re-evaluations that demoted their models, potentially leading to rapid iterations focused on private-set robustness.