AI Reimplements 16,000-Line Bioinformatics Toolkit in 14 Hours for $251
What's the largest software project AI can complete on its own?
Epoch AI and METR introduce MirrorCode, a benchmark testing AI on long-horizon coding tasks by reimplementing entire programs from scratch. With a generous inference budget, AI models work for days, and one model, Claude Opus 4.7, recreated gotree, a 16,000-line Go toolkit, in 14 hours at a cost of $251—a task estimated to take humans 2–17 weeks. The benchmark is cheat-resistant and includes held-out tests, though data contamination remains a concern.
We believe this same task would take a human engineer without AI assistance 2–17 weeks.
- Jdstanhope
I have been using Claude to build a clone of Bash in Rust (https://github.com/jdstanhope/huck). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 tests.
That being said, I am still guiding it on a regular basis and only recently let it run open ended trying fixing small to medium size bugs. The next stage will be try let it run in a loop trying to fix the 100 issues it has identified as well as getting most of the bash test suite to pass.
- gipp
"Reproduce an existing piece of software" doesn't seem like a problem that generalizes well to "develop a new piece of software," for a fair number of reasons.
- jph
I'm experimenting with large-ish software projects using Claude Fable. The results are surprising to me.
- Project portfolio management software. Good results so far. This is unguided experimentation: Fable researches feature ideas and plans them, then Opus 5 implements them, without me in the loop. The features include things such as an auth server, full-text search, audit logging, resource leveling, HA/DR, IaC, and more.
- FHIR & SNOMED medical software experimentation in demo-only environments. This has mixed results so far. This is guided experimentation: the FHIR specifications and SNOMED specifications are thorough and I'm in the loop. Fable plans OK, but Opus 5 is thrashing a lot, going down many rabbit holes.
- sreekanth850
We (Team of 3) built a retrieval engine from scratch in .NET for investigative search, with AI doing most of the backend implementation while humans handled the design and architecture and front end. It was developed using a task based, incremental approach. Includes more than 3,000 tests and extensive documentation, and is optimized for asynchronous ingestion using NATS and a disk backed WAL.
It includes a custom WAL, structure aware semantic chunking, ingestion and indexing pipelines, hybrid (rrf), text and vector retrieval, ACL aware filtering at query level, query classifier, citation tracking, and a CrateDB backend that performs fused retrieval over a single table.
total LOC: 180 K.
Status: evaluation by State agencies for investigative search on legacy documents.
tool used - Codex, Luna, and earlier codex 5.3, and gpt 4 mini.
- thisisauserid
I believe they can make ANY software project the largest it can be!
This solution should look for more problems.
- eithed
I'm looking at these reports and then at my sessions with Claude. In latest one, it got stuck in a loop fixing one thing and breaking another for 10 iterations, until I've noticed and gave it the insight. Or where it started continuously writing "echo", I don't know for what purpose. And I ask how, how can you not be the driver; how can you let these agents roam free and actually accomplish things.
- cyanregiment
Accurately? About a ~500 line React component or API endpoint given lots of very good context
Which is not bad
I don’t bother with the agentic stuff anymore than I would run a calculator on auto - what is the actual point if you think of what that really means
- manoDev
This actually proves the hardest part of software engineering is defining requirements.