Field note
MirrorCode changes the unit of coding evaluation from a patch to an entire executable program. An agent sees behavioral information but not the original source, then must reproduce the program closely enough to pass exact end-to-end tests, including held-out tests it never sees during development.
The benchmark is designed for long horizons rather than cheap samples. Epoch reports runs lasting up to 19 days and costing thousands of dollars; its current leaderboard can give each attempt seven days and a 10-billion-token budget. Sandboxing removes internet and source-code access, while three private targets remain unavailable to participants.
The caveat is equally important. The targets are open-source programs that models may have encountered in pretraining. Epoch's memorization screen reduces but does not eliminate contamination risk. MirrorCode therefore measures a useful capability under a disclosed uncertainty: whether agents can infer a whole behavioral contract and sustain implementation work across a software-project timescale.