Anthropic’s 64%: choosing a next step where humans went off course
A selected set of human detours, a separate control and an agent-designed project help distinguish suggesting a next step from choosing a research goal.
AnalysisExpand summaryCollapse summary
Mythos Preview’s 64% score comes from 129 deliberately selected human-detour moments. A separate sample of 127 strong human decisions yielded about 20% model preference, without per-model results. Agents also designed every experiment in a human-framed weak-to-strong project, recovering 97% of its defined performance gap; that result did not transfer cleanly to production-scale models. These findings show progress in research suggestions and methods, without demonstrating autonomous successor development.