Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
Recent developments indicate AI systems are now capable of automating the majority of engineering tasks in AI research, with research itself remaining less automated. This shift could reshape the landscape of AI development and research practices.
Recent benchmarks and research indicate that AI systems have achieved near-complete automation of core engineering tasks in AI research, while the automation of research itself remains uncertain.
Multiple independent benchmarks, including CORE-Bench and MLE-Bench, show AI systems are reaching or surpassing saturation levels in tasks crucial to AI engineering, such as reproducing research, optimizing models, and designing hardware kernels. For example, CORE-Bench, which measures the ability to reproduce research papers, reached a 95.5% success rate in December 2025, with some authors declaring it ‘solved.’ Similarly, MLE-Bench, assessing performance in Kaggle competitions, hit 64.4% in February 2026, equating to mid-tier human performance.
These advances suggest that the bottleneck for AI research shifts from engineering to the research process itself, which involves hypothesis generation, experimental design, and interpretive insights. The evidence indicates that AI’s capacity to handle engineering at scale is approaching or exceeding human-level performance, raising questions about the future role of human researchers in technical development.
Engineering Is Automated. Research Is the Residual.
Engineering / Residual · May 2026
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
● CORE-BENCH SOLVED 21.5% → 95.5% IN 15 MONTHS · BENCHMARK AUTHOR DECLARED IT COMPLETE
● MLE-BENCH PAUSED 16.9% → 64.4% · LEADERBOARD PAUSED APRIL 2026 FOR FAIR-COMPARISON REWORK
● POSTTRAINBENCH AI 25-28% VS HUMAN 51% · HALF HUMAN BASELINE · THE RECURSIVE TRIGGER
● RESIDUAL QUESTION ERDŐS 13/700 · 1 INTERESTING · MOVE 37 STILL UNREPLACED AFTER 10 YEARS
● ENGINEERING IS AUTOMATED RESEARCH IS THE RESIDUAL
● CPU SPEEDUP TASK 2.9× → 52× IN 11 MONTHS · 13× HUMAN BASELINE
● CORE-BENCH SOLVED 21.5% → 95.5% IN 15 MONTHS
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Source dossier · related dispatches
The Coding Singularity Outside Read · Piece 1 of the Outside Read series
Jack Clark Says It Out Loud · Clark Series 1
The Benchmark Saturation Cascade · Clark Series 2
The Compounding Error Problem · Clark Series 3
The Machine Economy · Clark Series 4
The Co-Founder’s Black Hole · Clark Series 5 Synthesis
The State of AI Replacing Jobs in 2026
Post-Labor Economics franchise
Jack Clark · Import AI 455 · “AI is getting good at core science skills” section · May 4, 2026
CORE-Bench paper · arxiv.org/abs/2409.11363 · September 2024
CORE-Bench “solved” announcement · Sayash Kapoor · December 2025
MLE-Bench · OpenAI · October 2024 · leaderboard paused April 2026
Anthropic CPU speedup trajectory · published in model cards · 2025-2026
Anthropic Automated Alignment Research proof-of-concept · Import AI #454
PostTrainBench · Import AI #449 · March 2026
Centaur math discovery · UBC/UNSW/Stanford/DeepMind · Import AI #441
Erdos problems · Aletheia (Gemini-based) · Import AI #444
Edison · “Genius is 1% inspiration and 99% perspiration”
AlphaGo Move 37 · DeepMind · 2016
Colophon
Set in Crimson Pro, Inter Tight, & JetBrains Mono. Composed for ThorstenMeyerAI.com, May 2026. Free to embed with attribution.
thorstenmeyerai.com
The outside read on Clark · second piece
Implications of Automated AI Engineering on Research Processes
The automation of engineering tasks in AI research could dramatically accelerate development cycles, reduce costs, and shift the role of human researchers toward higher-level hypothesis and theory work. However, it also raises concerns about the future structure of research teams, the need for new oversight mechanisms, and potential shifts in innovation dynamics. The institutional response should consider how to adapt to a landscape where engineering is largely automated, and research remains the primary frontier.
Recent Benchmark Progress and AI Capability Trajectory
Over the past 18 months, multiple benchmarks measuring AI capabilities relevant to research and engineering have shown rapid progress. CORE-Bench, focused on research reproduction, improved from 21.5% to 95.5%. MLE-Bench, testing Kaggle competition performance, advanced from 16.9% to 64.4%. These trends follow a pattern of saturation, indicating that AI systems are approaching or surpassing human-level proficiency in core engineering tasks essential for AI R&D. Prior to these developments, AI’s role was primarily supportive; now, it appears capable of automating substantial portions of the engineering process.
“The pattern across multiple benchmarks suggests that AI can now automate vast swaths, perhaps the entirety, of AI engineering.”
— Thorsten Meyer
Uncertain Extent of AI’s Research Automation Capabilities
While engineering tasks are increasingly automated, it remains unclear how much of the research process—such as hypothesis formulation, experimental interpretation, and theory development—can be automated. The structural question posed by Clark suggests that research may itself be a form of large-scale engineering, which could mean that research automation is closer than currently understood, but definitive evidence is lacking.
Next Steps in Monitoring AI’s Research and Engineering Progress
Researchers and institutions will need to observe ongoing benchmark developments, particularly in research-centric tasks. Further advances could lead to a redefinition of research workflows, with AI systems taking on more creative and interpretive roles. The next 12-24 months will be critical for assessing whether the residual research tasks can be automated, and how institutions adapt to this shift.
Key Questions
What does the automation of engineering mean for AI research teams?
If engineering tasks are largely automated, research teams may shift focus from technical implementation to higher-level hypothesis generation, strategy, and interpretation, potentially accelerating innovation.
Are there limits to AI automation in research?
Yes, it is still uncertain how much of the creative, interpretive, and hypothesis-driven aspects of research can be automated. Current benchmarks mainly measure engineering and implementation skills.
How soon could AI fully automate research processes?
While engineering automation is advancing rapidly, full automation of research tasks remains uncertain. Experts estimate this could take several years, depending on breakthroughs in AI’s creative and reasoning capabilities.
What are the risks of relying on AI for engineering and research?
Dependence on AI could lead to reduced human oversight, potential biases, and challenges in ensuring scientific rigor. Careful governance and validation will be necessary as capabilities expand.
Source: ThorstenMeyerAI.com