GPT-5.6 Sol Beats Factorio in 13.5 Hours, a First for AI Agents

What GPT-5.6 Sol actually did in Factorio

It launched a rocket. That is the win condition, and getting there took 13.5 hours of game time.

Factorio is not a game you finish by reacting quickly. It is a logistics puzzle: mine ore, smelt plates, build assemblers, route belts, and keep every stage fed without letting one bottleneck starve the next. Humans spend dozens of hours on a first run and still end up with a spaghetti factory.

Madden's setup let Sol play programmatically through a small Python library, but with the same restrictions a person at the keyboard would face: the player's reach, fog of war, and the rest of the GUI rules. The model was not reading the game's internal state and issuing cheat commands. It was playing, just through code instead of a mouse.

The run manager also logged production statistics throughout, annotated with science research milestones. Madden says he planned to stop after blue science, but Sol's effectiveness convinced him to keep going.

The catch: Sol papered over its own failures

The rocket launch was not a clean scaling story. It was a workaround.

Madden is blunt about it. The scaling issues he expected at the blue science stage showed up later instead, and they slowed Sol's relative progress significantly. To get the rocket off the ground, the model had to put in significant manual effort, repeatedly ferrying supplies between areas of the factory to cover for production systems it had failed to connect.

That is the interesting part. A human doing the same thing would call it a hack, and a bad one. For an agent, it is a sign that the model can recognize a goal is blocked and find a workaround, even when the workaround is inelegant.

It is also a reminder that 'beat the game' and 'play the game well' are different claims. Sol finished. It did not build a factory that would survive a stress test.

Why Factorio is a better AI benchmark than another chatbot score

Because you can watch it fail.

Madden makes the case himself: playing Factorio is in many ways analogous to programming, but far more legible to casual observers. A viewer can see a belt going nowhere or an assembler sitting idle. That makes it a convenient way to ground discussions of model behavior in something concrete.

The comparison to the previous attempt is telling. GPT-5.5, per Madden, failed to use transport belts at all. GPT-5.6 Sol not only used them but built a factory that produced a rocket. That is a jump in capability that a benchmark score would flatten into a number.

The broader agent race is moving in the same direction. Google's Genie 4 generated playable worlds and restarted the AGI argument; Factorio tests something narrower and more useful, whether a model can hold a long-horizon plan together without a human steering it. Madden says he is optimistic about autonomous Factorio play as a medium for building intuition about model capabilities, and plans to test other models, other thinking levels, and non-peaceful runs.

What it means for you

There is nothing to buy here. The run is a research demo, and Madden has not released the game manager or the Python library as a product. The full run is recorded, with a commentary video in the works and a timelapse already on YouTube.

For anyone in the US, UK or EU watching AI agents, the practical takeaway is the same: agents are getting better at long, multi-step tasks with a clear goal and a fixed set of rules. Factorio is a clean example of that shape. Most real work is messier.

Factorio was never going to be the test that proves an AI can run your logistics company. But it is a rare benchmark where the failure modes are visible to anyone watching a timelapse, and that makes it worth more than another leaderboard entry. The non-peaceful runs will be the ones to watch.