Qwen3.8 27B Scores 52 on Artificial Analysis Intelligence Index
Qwen3.8 27B, an open-weights model, achieves an Artificial Analysis Intelligence Index of 52, placing it among top performers. The index, based on nine evaluations including GDPval, Terminal-Bench, and GPQA Diamond, measures reasoning, knowledge, and agentic tool use. The model's performance is highlighted in a detailed breakdown of intelligence, cost, token usage, and context window, with comparisons against other models on cost and efficiency.
While model intelligence generally translates across use cases, specific evaluations may be more relevant for certain use cases.
- beltsazar
As a comparison, Qwen3.6 27B scores 38, which was the highest in its small model category (4B–40B).
Qwen3.8 27B beats all medium models (40B–150B). It has the same score as DeepSeek V4 Flash 0731, which ranks #5 in large model category (> 150B).
Sources:
- https://artificialanalysis.ai/models/open-source/small
- Balinares
And once again, Qwen 3.8 27B beats Opus 4.6, what the hell.
It's both funny and a bit terrifying and I still can't quite believe it. It runs decently on a gaming PC! Opus 4.6 came out only 6 months ago and was then broadly considered the new SOTA by a comfortable margin! How in hell did they package capability in the ballpark of a Feb 2026 frontier SOTA into 27B?!
More importantly, what's the point of building monster-scale data centers on unprecedented amounts of debt when a more than good enough model runs on a GPU from a couple years ago?
The coming months are going to be exciting, that's for sure...
- x313
I used this a lot over the weekend, and it's a really intelligent and strange model.
It gets really agentic at the higher reasoning levels. It does the basics like goal tracking and tool calling well, but more than that, it gets obsessed with solving problems and will do insane/unusual things to get to the solution. It actually reminds me of GPT-5.6-Sol-max which is similarly obsessive.
It doesn't surprise me at all that it outscores Opus 4.6. Opus had way better world knowledge but was more "human" with agent stuff - sort of lazy and uncreative, basically giving up once the obvious solutions failed. These newer models work more like magic, they are so creative and persistent at finding ways to get to the solution.
- K0IN
I used Qwen 3.6 27B extensively (>1B tokens) and DeepSeek V4 Flash (the older one also 2B+ tokens).
And I just can't fathom that the new 3.8 beats the new DeepSeek V4 Flash (which, in my eyes, is one of the best everyday coding models).
What an insane release, and convenient size to use every day/locally.
but i will test this model extensivly.
- kmike84
I have an internal automated benchmark, which roughly follows my workflow, and I've been testing various models on it, local and cloud. Qwen 3.8 27B did awesome. Its understanding is correct, research is better than e.g. glm's (and I like glm), and implementation is good and careful.
Qwen 3.8 27B doesn't look benchmaxxed. These "52 AA score" numbers feel real, which is surprising. I've been using it locally for a few days for other tasks as well. If not the speed, I'd be totally happy to use it as a daily driver instead of cloud models, it is that good.
---
(benchmark, to get an idea):
1. First, initial prompt which is not super precise - similar to how I'd write a task when talking e.g. to Opus. I'm describing an idea, and asking model to come up with some plan, and also to criticize the approach. Task is about implementing a particular pi extension. I'm checking if a model actually understands what I'm asking.
2. Then, as a follow-up, I ask to research alternative implementations, research UX of similar extensions, etc. It needs to do web searches, inspect open source codebases, read articles and papers, etc. I don't prompt to do this exactly, but I expect good models to figure out they need to do it.
3. Then, implementation.
Also, one finding: Q4 and Q8 seem to have very different behavior in this benchmark. Q4 produces 2-3x thinking in the end, and makes more turns - it seems it makes more mistakes, and needs effort to recover from them, while Q8 gets more things right in […]
- padolsey
The smaller these frontier-nearing models get, the more I'm reminded of https://en.wikipedia.org/wiki/Lottery_ticket_hypothesis
- RachelF
I have a bad feeling about this.
US companies have spent hundreds of billions on their models, and they are not much better than the cheaper open Chinese models.
Perhaps they can out-compete them. If not there will be increasing calls to limit access to open models on the grounds of "safety".
Basically, if you can't beat 'em, ban 'em.
- f311a
Why is it so small, but expensive?
Open Router
Input /M $0.45
Output /M $3.20
Cache read /M $0.05
Throughput 27 tps
It would be a very nice model at 200-300 tps and if it was dirt cheap.
What's the limiting factor of optimizing speed and price for inference providers?