MirrorCode shows AI models can recreate entire software from command-line access

METR and Epoch released MirrorCode, a metric that evaluates how well AI systems can reconstruct entire software applications using only command-line interface (CLI) access, without source code or internet connectivity. The metric, first announced in April, is now available in a full version with additional tests.

Main results

Out of 25 target applications, 17 were solved at least once to a full score, and four more achieved scores above 99 percent. Claude Opus 4.7 and GPT-5.5 both recreated gotree, a phylogenetic-tree analysis and manipulation tool containing 16 thousand lines of code, in several programming languages, with execution costs of $100 to $400 (about 370 to 1,480 shekels). Opus 4.7 also recreated pkl, Apple’s programmable configuration language with 61 thousand lines of code. The lowest reported cost is $251 for Opus 4.7 on a task that METR and Epoch estimated would take a human between two weeks and 17 weeks.

Remaining failures

Eight of the 25 applications never reached a full score, and four never passed the 99 percent threshold. The hardest software for the models was ruff, a linter and formatter for Python. The models also struggled particularly with giac_subset, a mathematics library, and mailauth, an email-authentication library—tools that require deep understanding of complex syntactic and semantic rules rather than generic logical reasoning.

Trend over time

The authors note that leading models from two years ago scored about 30 percent on the metric and were limited to simpler utilities such as calendar tools. The jump from 30 percent to full implementations for 17 of 25 applications, including one with 61 thousand lines of code, indicates a sharp improvement in long-term programming capability. The release version includes scaffold and 22 of the 25 target applications, totalling 132 task instances across six languages. Three target applications were omitted from the public release.

Why it matters

The authors stress that the significance lies not only in better code generation but in the models’ ability to operate in an unfamiliar environment and learn from it through limited access. In this case the environment is unknown software, and the model builds a functional replica from input-output behavior alone. They argue this hints at a future where AI agents can extract knowledge from the world and internalize it, creating parallel versions of technologies they encounter externally. The benchmark does not prove the broader claim, but it does demonstrate that practical software-cloning capability exists in this specific scenario.