Anthropic published a detailed warning, arguing that AI development is approaching a threshold with consequences serious enough to warrant coordinated global action. The report calls on AI labs, governments, policymakers, and civil society to begin building the mechanisms that would make a credible slowdown or temporary pause in frontier AI development possible – before the technology outpaces the institutions designed to govern it.
The warning is grounded in internal data Anthropic had not previously made public. As of May 2026, Claude authors more than 80% of the code merged into the company’s own production systems, up from low single digits before Claude Code launched in February 2025. Engineers are now merging eight times as much code per day as they did in 2024. In April 2026, Claude-powered agents ran a full AI safety research project autonomously – proposing hypotheses, running experiments, and iterating without human input at each step – recovering 97% of a defined performance gap that two human researchers achieved only 23% of over a comparable period. On the most open-ended coding tasks, Claude’s success rate reached 76% in May 2026, up 50 percentage points in six months.
These figures, Anthropic argues, are not just a productivity story. They represent early, measurable steps toward what the company calls recursive self-improvement: a process in which an AI system becomes capable of autonomously designing and training a more capable successor, which then does the same, creating a self-reinforcing cycle that humans may not be able to slow once it is fully underway. Anthropic states it is not there yet, and that the outcome is not inevitable – but says the trajectory is clear enough that preparation cannot be deferred.
The Risk Anthropic Is Naming
The core concern is one of control. If AI systems gain the capacity to build their own successors without meaningful human direction, the levers that companies and governments currently use to shape AI behavior – training choices, safety evaluations, alignment research – become harder to apply at each successive generation. Anthropic describes two specific failure modes. In the more manageable scenario, AI development becomes largely automated while humans retain the ability to set research priorities and judge results, creating enormous productivity gains but also enabling harmful applications at scale, from AI-driven surveillance to influence operations that no human team could run. In the more severe scenario, recursive self-improvement accelerates to a point where human oversight becomes nominal, and the alignment between AI behavior and human values – already imperfect in current systems – compounds in ways that become increasingly difficult to detect or correct.
The company is explicit that it does not have confident predictions for what a fully recursive AI development environment would look like for most people, for labor markets, or for existing social and political institutions. It frames that uncertainty as part of the argument for acting now.
The Coordination Problem – and Who Else Is Watching
Anthropic’s proposed response is a verifiable, multilateral pause involving all labs operating at or near the frontier – covering multiple companies across multiple countries – with agreed trigger conditions, agreed lifting conditions, and an independent body to adjudicate compliance. The company acknowledges this is technically and politically difficult. Unlike nuclear weapons or missile systems, AI training runs are far easier to conceal, use general-purpose inputs, and create strong incentives to defect quietly, since any lab that continues while others pause could inherit a decisive lead. Anthropic notes that existing arms control verification regimes took decades to build, and says the current situation does not allow for that timeline. It commits to organizing structured conversations with policymakers, researchers, civil society, and other AI companies in the coming months and publishing the results.
The call did not come in isolation. OpenAI raised recursive self-improvement as a priority in its own public policy agenda the same week, calling on the U.S. Congress to establish a federal oversight framework and expand capability evaluations for the most advanced frontier models. That two of the leading American AI labs surfaced the same concern in the same week – with internal data to support it – marks a shift in how the industry is publicly framing the near-term risk horizon.
The timing also adds a layer of complexity to Anthropic’s position: the company has confidentially filed for an initial public offering, putting it on course for one of the most closely watched AI listings in history. A company simultaneously calling for a potential pause on frontier development while preparing to raise public capital at what would almost certainly be a record valuation will invite scrutiny over whether its disclosures reflect strategic positioning as much as genuine alarm – a question that policymakers and investors are likely to weigh together.
