My personal AI benchmark: "Generate an SVG of a frog with a Habsburg jaw."
A developer's personal benchmark pits AI models against a single prompt: generate an SVG of a frog with a Habsburg jaw. Each model gets three tries a month. The site tracks 14 models and 42 runs, all producing SVGs. The results showcase how different models interpret the prompt, with some adding editorializing comments about the frog's exaggerated jaw.
The annotations are mostly structural labels, but include some editorializing about the jaw feature: "massive protruding mandible" and describing the upper lip as "recessed, tucked behind the jaw" and lower teeth as "protruding" over the upper lip, which offer anatomical interpretation beyond plain labeling.
- jnwatson
Fable 5 on Max knocks it out of the park:
Definitely has some creative flourishes.
(I made no extra prompting. Just the above text. Single shot.)
- hn_throwaway_99
I thought this was great, and hilarious. Kudos to Opus 5, I thought it was the only one that came close to passing. Interestingly, I thought many of the failures drew the frog face OK, and they had some type of big blob for the jaw, so they knew "Hapsburg jaw" meant a protruding jaw, but it wasn't really connected to the frog face in any way that made sense.
Small side note, the first gemini-2.5-pro one totally reminded me of some sad faced meme or Pepe the frog from somewhere. Anyone know what I'm referring to, tried to find it.
- krisoft
Curious that none of the attempts draw the frog from side profile. If i have to draw this i would immediately know that drawing a recognisable frog is the easy part of job. Expressing a particular jaw shape and melding it on the frog is the hard part. And jaw shapes are more prominent from the side.
Even absence of thinking this through you would think that some frogs will be from the front, some from the side. Just by chance. And yet all appears to go for the harder pose.
- abound
My personal benchmark is a directory containing a bunch of research papers on the physics of popping popcorn kernels, and a prompt about creating high fidelity, photo realistic 3D models of all the different kinds of popped kernels. Fable (surprisingly? unsurprisingly?) refused to do it last time I tried, and the results from other models are, well, fine, but there's still plenty of headroom on this particular one.
- thebigship
Hi all, the site is getting hugged to death, thank you, was not expecting this kind of warm response. I will be working to make this more reliable, in the meantime, sign up for my newsletter: https://www.jaymollica.com/blog/
also my favorite SVG was def the google/gemini-3.6-flash
edit: ok better now I think