GPT-5.6 Sol beats Factorio in 13.5 hours of game time
GPT-5.6 Sol can beat Factorio on peaceful mode
Ryan Madden ran GPT-5.6 Sol in Codex at a low thinking level and let it play peaceful Factorio through a Python game manager that enforces human-like limits such as reach and fog of war. It launched a rocket after 13.5 hours of game time, far better than GPT-5.5, which never learned to use transport belts. Late-game scaling problems forced Sol to ferry supplies by hand to paper over broken production links.
The rocket launch was only achieved through significant 'manual' effort on Sol's part as it papered over its late-game failures to connect certain production systems by repeatedly ferrying supplies between areas of the factory.
- mwt
> The rocket launch was only achieved through significant ‘manual’ effort on Sol’s part as it papered over its late-game failures to connect certain production systems by repeatedly ferrying supplies between areas of the factory.
Robots: they're just like us!
- minimaxir
If you haven't been keeping an eye on LLMs playing games, GPT 6.1 Sol can new beat games 100% vision-only (without harnesses that give the agent extra information about the game). Pokemon is no longer a difficult challenge for it, even with additional difficulty restrictions such as hard mode and level caps. Recently, GPT 6.1 Sol beat Final Fantasy VII and its bonus bosses vision-only surprisingly quickly.
I suspect Opus 5.5 can do even better but there aren't as many people testing it.
- mickeyp
Watching AI play games is fun, but I can't wait for someone to crack having small LLMs in games to enrich them in useful and interesting ways.
- program_whiz
One limitation of these type of demos (and similar with Minecraft, etc) is that:
1. Putting into "peaceful" mode makes response time a non-factor
2. These models have read and have access to every guide and video ever published on the subject. Most humans, especially on first play, learn game systems and interactions from scratch, then develop novel strategies and plans and "hypothesis test" their way into beating the game.
This is still impressive compared to where we were a few years ago, and being able to synthesize all existing information about a game and then play it to completion in a semi-realistic setting is great.
At a quick glance someone might mistake this for "brilliant AI solves Factorio", when in reality, given a similar but different game where it wasn't gifted all the information or training data (e.g. massive videos and screenshots) it might fare terribly (as others have pointed out in comments).
- Majromax
> The rocket launch was only achieved through significant ‘manual’ effort on Sol’s part as it papered over its late-game failures to connect certain production systems by repeatedly ferrying supplies between areas of the factory.
Looking through the video, around 12m one can see the Sol trace report the cancellation of a conflicting 'courier' (hand-feeding) script.
Rather than implement 'manual' effort, it seems like Sol considered the Engineer to be just another unit of automation!
That's not exactly within the intent of the challenge, but it's not too far out of bounds either. It's also a hard loophole to close if the agent interacts with the game through a scripting layer.
- fasterik
It doesn't seem to understand the core mechanic of Factorio, which is putting outputs onto belts and routing them to other inputs. It does this in a few places, but for the most part everything is put into chests and moved around by the player.
- VoidWhisperer
Factorio is a much easier game without the bugs being a pain in the arse as you expand - i wonder how Sol would handle the bugs
- kanemcgrath
It is weird and almost alien how AI designs factories. When Claude announced Fable 5 they also did a Factorio demo that was the same kind of designs and a lot of hand feeding and direct insertion.
In some sense it proves to me that llms are not just copying and are actually creating a bespoke solution. But I honestly expected more from frontier models