11 AI models, one prompt: which is worth your credits?
Choosing an AI model: one prompt, 11 models, different results
Netlify now offers access to 11 AI models via its Agent Runners and AI Gateway, including new open models like Kimi K3, GLM 5.2, and DeepSeek V4. To help users choose, they ran identical prompts across all models and compared results, credit usage, and quality. The test reveals a wide cost range—from 2.4 credits for DeepSeek V4 Flash to 519 for Claude Opus 5—with surprising differences in output quality. Opus occasionally overspends but delivers richer designs, while cheaper models like GPT 5.6 Terra offer a different, not necessarily worse, visual style. The full report is available online.
That’s a pretty wide distribution, eh? Not only that: the Claude Opus average is heavily slanted upwards because one of its three runs spent a whopping 1,055 credits!
- Systemerror7A69
Am I wrong or are these evaluations, while interesting, not really meaningful for anyone doing serious development work?
I'm asking because I personally only use AI with specific and detailed instructions, building my projects piece-by-piece.
I mostly don't look at the low level code and some of it I don't understand as much as I'd like, but I very much give much more technical instructions than a simple, two sentence prompt.
So, it seems to me this "oneshot from a simple prompt" eval is fairly meaningless when it comes to model evaluation itself, as it is in no way representative of real world application.
This would be more something for "vibe coders", people with little to no programming background wanting a website?
- isqueiros
> Build a one-page site for a neighbourhood coffee shop: opening hours, the address, a short menu and a photo. Nothing on it changes unless I edit it myself.
If that's the entire prompt, it's quite depressing how much alike these all look. I appreciate some of the details from the Opus 5 version, but I can't help but strongly feel the AI vibes emanating from that design.
- 44za12
Built rightsize exactly for this. https://nehmeailabs.com/right-size
- jwr
I've been doing a lot of benchmarking for a long time now with a number of local models for the purposes of spam filtering. The major observation is that there is a lot of variance in model performance. This should not be surprising, as these are probabilistic machines based on random numbers, so your performance will vary from run to run. But this also means that any sort of evaluation of benchmark with a sample size of 1 is essentially worthless for the purposes of model comparison.
In my benchmarks, I started insisting on having at least 5 runs.
This also makes me very suspicious of some of the posted benchmarks that compare models. If the results don't say how many benchmark runs were performed and what the variance was, there is really no basis for comparison. You are just guessing.
- sinuhe69
Having skimmed through the article, I assumed that they hadn't tested the design for mobile devices because it wasn't mentioned. That would be a huge mistake. In today's web design, a mobile-first approach is imperative. This is even more important if you want to showcase your café.
I used the developer tool in Firefox to see how this design would behave on mobile devices. There are huge differences.
Some designs use the screen estate so ineffectively that only the title and a big, boring, generic graphic is shown on a phone. Users have to scroll all the way down to see the content and find what they need. Better designs show the menu, navigation points, and meaningful, aesthetic graphics. Other designs, such as Gemini 3.6, were quite sophisticated but not optimized for traffic and would not load on a 3G connection. However, a simple static website should load instantly on a mobile connection.
That said, even a simple web page has many requirements, so expecting a turnkey, ready-made design if the user is not guiding the process is not realistic. Thus, I believe the best choice nowadays is a model with good design skills that understands and adheres to an iterative design process, offering a good initial design as a starting point but also prompting the user to provide guidance and feedback. As the design process runs through many cycles, the initial cost should be modest. But more importantly, the model should understand its own design, be able to explain its choices so it c […]