GPT-5.6 Sol: OpenAI's Best Vision Model Yet, But Gemini 3.5 Flash Still Leads
GPT 5.6 Sol is the best "vision" model OpenAI ever released
OpenAI's latest GPT-5.6 lineup, including Sol, Terra, and Luna, shows significant vision improvements, especially in object detection and counting. Sol scored 46.2 mAP@50 in detection, up from 13.8 for GPT-5.5, and improved counting accuracy to 73.0%. However, Sol lags in OCR and text extraction, and its higher token usage and cost make Gemini 3.5 Flash a more practical choice for high-volume tasks. The models also struggle with large images and certain complex scenes, but overall, OpenAI is now a stronger contender in vision.
Sol is clearly the best vision model OpenAI has released so far.
- HarHarVeryFunny
The summary "There are still clear limits. Gemini 3.5 Flash remains a better practical choice [than GPT 5.6 Sol] for high-volume detection and counting in our benchmark, especially at its price." seems rather understated !
GPT 5.6 Sol was outperformed on all benchmarks by Gemini 3.5 Flash, apart from a single exception (OCR) where Fable was the winner.
Gemini 3.5 Flash not only outperformed GPT 5.6 Sol, but did so at 1/3 of the cost.
- weli
Anecdotal, opinion:
Gpt is really good in vision stuff, or at least their MoE seems to be really cohesive. From my experience Claude models can be really good at language but the moment they need to look at a picture and decide why the design is not good what parts need improvement it degrades a lot. My easiest benchmark is giving them a screenshot of a feature in my app and tell it "identify non-normative UI blocks and improve readability and consistency". Sol does a great job at re-structuring the page into composable units that build upon each other and the general looks and feels of the app. Claude tends to over-focus one one part while completely forgetting about the rest or the cohesion as a whole.
- evrimoztamur
Penny sample shown looks like failed EXIF orientation registered by the model/harness. The coins are correctly marked, it's rotated 90 degrees.
- bearjaws
It is funny to me seeing Sol used for what a "traditional" AI model can do already (counting pills).
We have vision models for our pharmacy and I could never imagine taking the latency hit to use a Sol in our robotics, it would be likely 25-50x slower.
- dllu
Vision is still embarrassingly bad.
ChatGPT Pro with GPT 5.6-sol: https://chatgpt.com/share/6a834217-ca8c-83e8-a8e8-45d5b8797b...
The puzzle: https://activityvillage-files.s3.eu-west-2.amazonaws.com/s3f...
- fpgaminer
Gemini 3 Flash should really be included in this comparison. Or at least 3.7. In most of my testing, 3.5 and 3.6 were both a downgrade in terms of vision capabilities, relative to 3, and at a much higher cost. 3.7 is slightly better than 3, finally.
- mv4
Ironically, the pill counting example selected to showcase "the best vision model" can be easily solved with OpenCV template matching, a technology created 25 years ago.
- kzrdude
In the third vision bench result, Sol is 100% correct but the expected has 1 error. Seems like an oversight.
In the next bench, Sol looks like it’s correct again but the bboxes are rotated 90 degrees for some reason.