Why some AI labs now want to slow down: Changes in model training have led to rapid gains 

President Trump and Anthropic CEO Dario Amodei had dinner at the White House on Sunday, reportedly their first one-on-one meeting. While neither side has said what they discussed, AI safety was likely a focus. The dinner came two weeks after Amodei urged AI labs to slow the pace of model development, and after Trump answered him on Truth Social, essentially disputing the need for any such slowdown. Recently, NVIDIA CEO Jensen Huang has referred to the calls for a slowdown as “odd.”

In any event, the calls for such a slowdown have grown in September 2026, following the resignation of Anthropic researcher Jacob Coxon, who claimed top AI labs are “gambling with our lives” by racing toward uncontrollable, self-improving superintelligence.

In addition, Amodei recently published an essay saying that the industry must “pace the frontier,” following a series of high-profile incidents, most notably one this summer in which hundreds of OpenAI agents targeted Hugging Face, a high-profile repository for AI models and datasets used to train them.

In recent weeks, Trump called fears of AI “taking over the World, destroying Humanity” a hoax and wrote that the only guardrail AI needs is a “strong and smart” president. Trump cast opposition to AI and data centers as a “conspiracy” that helps only China.

Part of the incentive to keep the AI engine filing on all cylinders is financial. Trump referred to AI in a follow-up post as the Golden Goose. Several economists have essentially said the same thing in more technical terms. U.S. real growth ran around 1.5–2.1% in the first half of 2026. Torsten Sløk, partner and chief Economist at Apollo, an asset management firm, and others have said AI-linked spending is on the order of half of 2026 U.S. growth, or about 1 percentage point of GDP growth from data centers, chips, power and construction. Moody’s chief economist Mark Zandi estimates that AI and related spending account for about a quarter of U.S. economic growth, and seven AI-heavy companies make up about a third of the S&P 500’s value, according to Newsweek. The public, meanwhile, is uneasy: 70% of U.S. adults told NBC News they are more worried than excited about AI.

The policy debate reaches Washington

The U.S. already has federal AI policy on the books. Executive Order 14409, signed June 2 after a postponed signing ceremony, sets up a voluntary pre-release review of frontier models focused on cybersecurity. Commerce Department export controls briefly forced Anthropic to pull its newest models this summer. The open question is what comes next: Vice President JD Vance says the administration wants to “regulate smartly,” but he has called regulation requested by the AI companies themselves a possible “Trojan horse.”

Now, the frontier AI labs themselves appear to be calling for more scrutiny. Earlier in September, OpenAI posted an article titled “The AI policy window is open. We need to act,” that lobbies for mandatory national AI safety requirements.

Public reports of models cheating, escaping their sandboxes and the like have grown more commonplace as the pace of model development has ramped up, leading frontier model labs like OpenAI and Anthropic to delay or modify plans for model releases. The shift involves a change in how frontier labs are training models and an increasingly agentic dimension to model development, a trend that has gained steam this year. By Anthropic’s count, Claude now “leads” 26% of the company’s AI R&D work. Here, “leads” is a technical term Anthropic based on concepts from Epoch AI, the research organization.

From model training to AI-led research

OpenAI said on August 18 that it had paused reinforcement-learning training on its newest models for two weeks to harden and red-team its research environments, and that its largest planned frontier training run remained on hold. Anthropic said it had frozen changes to its training environments for about a month in April, after flagging more than 10% of them for problems including reward hacking, in which a model finds a way to earn a high score without completing the task.

In essence, labs now reward models for solving multistep tasks with software tools, while using AI agents to help design, run and assess research. The sidebar traces the main training changes.

Jack Clark, an Anthropic co-founder, put the odds of a system that can train a more powerful successor with no human intervention at roughly 60% by the end of 2028. OpenAI is aiming for an automated AI researcher by March 2028, and this month the company said it was making “strong progress” toward that goal.

Meta CEO Mark Zuckerberg treats the same step as a matter of competition. In his essay “The Future is for Everyone,” he describes a dilemma that arises “once AI systems can autonomously improve themselves”: “any lab that doesn’t let their AI system direct a substantial amount of compute capacity towards recursive self-improvement will inherently fall behind.” A self-improving system focused on compute efficiency, he wrote, “could theoretically invent ways to squeeze 100x or more intelligence out of each gigawatt.”

A wider group of researchers also expects a short timeline. A Cambridge working paper by more than 20 academics and AI lab researchers says AI systems are on track to automate most AI R&D work within a few years, possibly all of it, and that some extrapolations of recent trends suggest AI could automate months-long AI R&D projects by mid-2028. The authors include OpenAI’s Jakub Pachocki, Anthropic’s Jack Clark and Microsoft’s Eric Horvitz, along with Nobel Prize winner Geoffrey Hinton and A.M. Turing Award winner Yoshua Bengio.

Some researchers question the timeline

Other researchers doubt that AI will be able to improve itself so soon. In a study from researchers at Princeton University, AI agents received research questions from two unpublished papers submitted to NeurIPS 2026, a top machine-learning conference, and were asked to produce publishable papers. The agents handled the engineering, reviewed the literature and ran hundreds of experiments. The papers’ original authors, grading the results as conference reviewers would, rejected both.

The agents lacked the judgment and creativity that research requires, one researcher told MIT Technology Review. He traced the gap to training: reinforcement learning works best on tasks a computer can check automatically, and open-ended research doesn’t fit that mold.

Clark acknowledged the findings in his newsletter, calling the study “a somewhat bearish signal on short recursive self-improvement timelines” and writing that such papers “continue to show that there’s a certain absence of valuable, intuitive creativity in today’s AI systems.”