Open-weights models fall 12 points behind the frontier as Artificial Analysis launches Intelligence Index v5
Artificial Analysis Intelligence Index v5
Artificial Analysis overhauled its flagship Intelligence Index, with 60% of v5 now built on private or held-out test sets. Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55). The open-weights gap widened from 3 points to 12: Kimi K3.1 Pro tops open models at 46, while released-weights leaders DeepSeek V4.1 Flash 1017 and GLM-5.3 sit at 32. Cost per task spans more than 13x among top models.
The open weights gap widens from 3 points to 12.
- peri-cl
The writeup fails to disclose that they quietly redacted several open-weight models from the new benchmark. Popular ones. They're not on the charts; they're not available to manually add to the charts; and their individual model pages now display scores of "N/A".
GLM 5.3-flash: N/A
https://artificialanalysis.ai/models/glm-5-3-flash/
Qwen 3.8-flash-next: N/A
- simonjgreen
This might be the first time I’ve favourited a post without upvoting it.
This is really sus. I’d love a more detailed explanation and “take” from AA on what problem they felt needed correcting and why things have shuffled the way they have. And how they can claim their index has relevance if positions can be switched around so easily.
- kamov
The natural language choice in multilingual index seems pretty strange and biased to me, they tried to encompass a broad swath of Asian languages, but then there's like four European romance languages. They didn't include any Slavic one i.e Russian, or Nordic one like Swedish, or Finno-Ugric. And these romance ones are probably overrepresented in the training data anyways
- chvid
"The FrontierCode verifier grades each attempt with the task's tests and rubric. We use GPT-6 Luna (medium) as the LLM judge"
Why are they using a closed source LLM as a judge? Closed source providers can see your traffic and adjust as they see fit.
- lousken
It is embarrassing how much they suck up to american labs while those models only perform the first few days after release. With open weights you can be sure the quality stays the same unlike the closed source crap. How about a retest after some time?