AI Benchmarks & Leaderboard — 2026-09-16

New Model Releases & Updates

Agent Effectiveness Index (AEI) by Open Source Community

• Type: Benchmark
• Key benchmarks: Scores AI agents on their ability to learn and retain complex tasks from human demonstrations.
• vs. Previous best: Provides a new metric for agent effectiveness beyond traditional static benchmarks.
• What's notable: The AEI is an open-source initiative aimed at standardizing how agent capabilities are measured in dynamic environments.

2 sources

globenewswire.com

globenewswire.com

ml.globenewswire.com

ml.globenewswire.com

Free 744B Agent (Unnamed) by Community/Independent Developers

• Type: Open-source model
• Key benchmarks: Outperformed GPT-5.6 Sol on the BrowseComp benchmark.
• vs. Previous best: Challenges closed-source frontier models in web navigation and browsing tasks.
• What's notable: Highlights the rapid progress of open-weight models in specialized agentic tasks.

Leaderboard Snapshot

Frontier Models (Closed-Source)

Note: Rankings based on Artificial Analysis Intelligence Index and speed metrics.

Open-Source Leaders

Note: Specific scores for open-source models were not explicitly detailed in the recent changelog snippet, but these models were noted as new or updated entries.

Benchmark Deep Dive: The Agent Effectiveness Index (AEI)

The release of the Agent Effectiveness Index (AEI) marks a pivotal moment for agentic AI evaluation. Unlike traditional benchmarks that test static knowledge retrieval or single-step reasoning, the AEI focuses on an agent's ability to learn complex workflows from human demonstrations and retain them over time. This shift addresses a critical gap in current evaluation methodologies, which often fail to capture the iterative, interactive nature of real-world AI agent deployment.

The benchmark evaluates models on their capacity to observe human actions, generalize the underlying logic, and execute similar tasks autonomously. This "learning-by-demonstration" capability is crucial for scaling AI utility in domains like software development, customer support, and physical robotics. Early results suggest that while frontier closed-source models lead in raw reasoning, the efficiency of learning from limited demonstrations remains a key differentiator where open-source models are showing competitive promise.

For practitioners, the AEI provides a more practical metric for selecting agents for enterprise applications. It moves beyond "can it answer this question?" to "can it learn how to do this job?" This aligns with the industry's growing focus on autonomous agents that can handle unstructured, multi-step tasks without constant human intervention.

Analysis & Trends

• State of the art: Claude Fable 5.1 currently holds the top spot on the Artificial Analysis Intelligence Index, followed by GPT-6 Astra. Speed leaders include Celeris-1 and Mercury 2, indicating a bifurcation between pure intelligence and inference speed.
• Open vs. Closed gap: The gap is narrowing in specific agentic tasks. A free 744B parameter agent recently surpassed GPT-5.6 Sol on BrowseComp, suggesting that open-source models are becoming highly competitive in web-based agent benchmarks.
• Emerging patterns: There is a clear trend toward specialized benchmarks like AEI and BrowseComp, moving away from generic MMLU/GPQA scores toward task-specific agentic capabilities.

What to Watch Next

• AEI Adoption: Monitor how quickly the Agent Effectiveness Index is adopted by major labs and whether it reveals weaknesses in current frontier models' learning capabilities.
• Open-Source Agentic Performance: Track further releases of large open-weight agents (like the 744B model mentioned) to see if they can maintain performance advantages on BrowseComp against upcoming closed-source updates.
• Speed vs. Intelligence Trade-offs: Watch for new models that attempt to bridge the gap between the high intelligence of Claude Fable 5.1 and the high speed of Celeris-1, as efficiency becomes a primary cost driver.