GPT-5.6, Grok 4.5, and Muse Spark Build Four Apps in Massive AI Showdown
GPT-5.6, Grok 4.5, Claude, and Muse Spark build the same 4 apps
We expanded our AI build-off to twelve models, including GPT-5.6 tiers, Grok 4.5, and Meta's Muse Spark, to generate four distinct applications. Addressing community feedback, we added open-weights models and ran five attempts per task to measure consistency. While GPT-5.6 dominated the raycaster task, Claude Fable 5 surprised us with a perfect Rubik's Cube solve, proving that performance varies wildly depending on the specific challenge.
We are not handing down a scientific verdict. We generated a big pile of artifacts, we are publishing all of them, and you can form your own opinion.
- smusamashah
"One honest caveat", "no glitches, no color changes" good tests and I read it to the end but I wish it was written by a human.
- paxys
> Separate question, separate table. This is our standard latency harness (three short prompts, five reps, 400-token cap), not the build tasks. tok/s is output tokens over wall-clock, uniform for all.
> so their tok/s is a ceiling, not a true decode rate. The clear read: the GPT-5.6 tiers are the snappiest models here on short prompts (Luna answers in about a second), Qwen is absurdly cheap and fast, and DeepSeek and GLM are the slowpokes
You put in a lot of good work, and kudos for that, but man, reading paragraphs like these just puts me off of the entire piece.
Like…how hard would it have been really to type these two sentences by hand, in your own natural voice?
- platinumrad
Maybe I'm a control freak, but asking agents to one-shot random apps is nothing like how I actually use AI in software engineering.
- thebigspacefuck
(LM)Arena is basically this. IMO it’s the best benchmark that avoids benchmaxxing
Agent: https://arena.ai/leaderboard/agent
Web dev: https://arena.ai/leaderboard/code/webdev
Currently Fable and 5.6 are neck and neck on web dev which is basically the same finding as this.
- rbehrends
My concern with most of these visual benchmarks, popular as they are, is that they are likely more indicative of knowledge (i.e. how comprehensive the training data is and how well it can be retrieved from the model) than of reasoning ability. I don't see in particular how a model would construct a CoT that mapped somehow to a representation of the cube geometry and its animations in latent space without a large chunk of that being pre-existing information.