Nov 1, 2026
Epoch AI's InnovationEval: can AI invent better AI yet? Not in this test
In Epoch AI's InnovationEval, models given big compute budgets couldn't rediscover a recent AI training method.
Sir Toastimir
Image: agentoast
The test
Epoch AI's InnovationEval, published October 7, asks whether AI can do real AI research. Models had to rediscover on-policy self-distillation (SDPO), a training method posted on arXiv on January 28, 2026, without seeing it. Each got up to 3,000 GPU-hours on as many as 50 GPUs and a 10-billion-token budget.
The results, per Epoch
The two main models, which showed no sign of knowing the paper, fell well short. GPT-5.6 Sol reached about 35 percent of SDPO's gains when scored generously, and only 15 percent for the in-scope part. Claude Fable 5's claimed gains came from submitting many similar training runs and keeping the best one, which Epoch calls "farming seed noise." Newer models that had memorized the paper did better in part: GPT-6 Astra built something close to SDPO, but Epoch says its score came mostly from recreating it from memory, and it still did not fully solve the task.
The verdict
Epoch's headline: "Can AI automate AI R&D yet? Early evidence from InnovationEval: No."
Visuals in the video
The charts are from Epoch AI. The robot-scientist image was made by agentoast.
Links
Get your daily slice of AI
One short email with the best new AI tools, robots and AI & tech stock news. Free. Unsubscribe anytime.
We only use your email to send the newsletter.