Its suite contains 11 tasks in five categories, including graphic design, data insight generation, data explorer creation, data center research, and research design. Six models received context and tools and attempted the tasks without further intervention; Epoch manually graded outputs against its employee standards. The report finds that frontier models can perform reliably on well-defined tasks while falling short of autonomous Epoch-quality work.
Implication: A strong score on a well-defined task should not be treated as evidence that a system can autonomously own an open-ended workflow.
Thesis
Real-work automation depends on judgment against local standards and on the ability to validate a result, not just on completing a bounded task.
Observed facts
Epoch says its task suite contains 11 tasks across five categories, including graphic design and research design.
1Six models attempted tasks with supplied context and tools; Epoch manually graded outputs against employee standards.
1The report says frontier models performed reliably on well-defined tasks but failed in more open-ended aspects that prevent full automation.
1
Mechanism
Epoch combines messy, organization-specific tasks, tool use, and manual grading against employee standards, exposing gaps that easily verified benchmark tasks can miss.
Why now
As organizations assess whether frontier models can take on work, task suites built around realistic context and evaluation standards test a different capability boundary than simplified, automatically checked benchmarks.
Implications
A strong score on a well-defined task should not be treated as evidence that a system can autonomously own an open-ended workflow.
Evaluation should test whether an agent can meet local quality standards and validate its own experimental claims.
Evidence boundary
This is an evaluation of 11 Epoch tasks, graded against Epoch’s standards; it is not a general estimate of automation across occupations, companies, or all possible workflows.
Watch conditions
Further task suites should show whether these gaps persist across organizations and whether models can improve when evaluation criteria and feedback loops are made explicit.