An Anthropic Researcher Left Over AI Risk. What His Exit Really Means

The Exit That Isn’t Just an Exit

Jacob Coxson is 27, British, and he spent the first months of this year at OpenAI before moving to Anthropic — not because he was disillusioned, but because he believed Anthropic’s safety culture offered a better shot at building responsible AI. By September, he was gone again, this time for a very different reason.

In a post on X, Coxson warned that the race to build self-improving AI systems was pushing safety to the margins. He told the Wall Street Journal that most developers in the field genuinely believe the technology could kill all of humanity within a decade, and that next year could already be too late to regain control.

That alone would make headlines. What makes this story unusual is what happened next.

The Inside Corroboration

Evan Hubinger, Anthropic’s head of Alignment Science, did not distance himself. He said Coxson was right. He added his own probability estimate: more than 10 percent chance of human extinction from AI within ten years.

Samuel Marks, who leads Anthropic’s Scalable Oversight team, issued a nearly identical assessment — again, framing it as a personal view, which in this context is as close to an institutional acknowledgment as the company is likely to get. Marks went further, noting a pattern he found striking: the higher the rank of the AI developer, the more likely they were to express serious concern about existential risk.

Yet development continued anyway.

Marks attributed this contradiction to two forces. The first is commercial incentive — companies that slow down risk falling behind. The second is a more pragmatic fear: that if responsible actors pull back, bad actors will accelerate, and the outcome will be worse.

This is the framework that animates much of the current AI safety debate. It is also a framework that leaves ordinary people with very little leverage.

Why One Resignation Carries Weight

Coxson’s departure is not the start of a mass exodus from Anthropic. But it is significant precisely because of who he is and where he came from. He did not arrive at Anthropic as a critic. He joined in January, attracted by its stated commitment to alignment research. His exit is therefore not the story of an outsider protesting from the sidelines — it is the story of someone who went inside and concluded that the structure itself was the problem.

His critique of both OpenAI and Anthropic is blunt. He told reporters that neither company is acting responsibly, describing the industry’s rush toward superintelligence as a gamble with human lives. He called the fact that decisions about technology capable of reshaping civilization rest with a small group of engineers a kind of madness.

The distinction matters. When critics outside the industry raise these alarms, they are easy to dismiss as Luddites or attention-seekers. When a 27-year-old researcher who chose Anthropic specifically for its safety orientation reaches the same conclusion, the narrative shifts.

The Signatories Who Stayed Behind

Coxson is not alone in his concerns. Earlier this year, he signed a statement calling for governments to build international frameworks that could slow the pace of AI development. The signatories included more than a thousand researchers, among them Anthropic CEO Dario Amodei and OpenAI chief scientist Jakub Pachocki.

That list is remarkable for what it contains and what it conceals. Amodei and Pachocki are leading the very companies Coxson now says are gambling with humanity’s future. Their signature on a cautionary statement does not necessarily mean they are slowing down. It means the language of caution has entered the industry’s DNA — even as the engines keep running.

What the signatories document is less a coordinated call for restraint and more a shared vocabulary of alarm. The same leaders who signed the statement are simultaneously overseeing the fastest iteration cycles in the company’s history. That gap between public positioning and operational tempo is where the real story lives.

The Second-Order Effects Nobody Is Talking About

The most consequential impact of Coxson’s departure may not be immediate. It is subtler, and it plays out in the years ahead.

First, there is the recruiting signal. Anthropic has spent years cultivating a reputation as the company where safety researchers can do their best work without being pressured into shipping faster. If junior researchers begin to see that even the safest-looking environment cannot absorb honest warnings about existential risk, that narrative loses traction. The company that built its brand on trust is now being asked to explain why its trust did not prevent a researcher from leaving over it.

Second, there is the media feedback loop. Coxson’s story will not be the last. Each departure from a top safety lab that cites existential risk as the reason adds a layer of credibility to the next one. The argument that these warnings are fringe or exaggerated becomes harder to sustain when the warners are people who actively chose safety-first labs over faster competitors. That erosion of skepticism is not something any company can easily contain.

Third, and perhaps most quietly, there is the investor relationship. Anthropic raised $6.4 billion from Amazon and SoftBank in 2024 on the promise of leadership in safe AI. If the market begins to price existential risk as a material factor in valuation — rather than as PR language — the financial architecture around these companies starts to shift. Capital allocators are not yet pricing this in, but the framework exists. The question is only timing.

The Geopolitical Layer

The timing of Coxson’s warning lands in an especially volatile moment. China has entered the AI development race with aggressive state support, raising the stakes for any company or country that considers pulling back. The Trump administration has signaled a preference for minimal regulation, which removes the primary external check on development speed.

For readers outside the United States, this has direct consequences. The policy environment in Washington shapes the global competitive landscape. If the U.S. opts for deregulation while other nations impose constraints, the incentive structure tilts sharply toward speed over caution — exactly the dynamic Coxson described.

But the geopolitical dimension cuts both ways. A U.S. retreat from AI regulation does not mean the rest of the world will follow. The European Union has already moved toward binding rules with the AI Act. China’s state-driven model creates its own pressures for acceleration. The result is a fragmented regulatory landscape that makes coordination — exactly what Coxson and the signatories called for — exponentially harder to achieve.

What Happens Next

The practical question this resignation raises is not whether Anthropic will lose more researchers. It is whether the company’s leadership will allow internal dissent to translate into public position statements that match the private conversations Hubinger and Marks have been having.

Anthropic has consistently positioned itself as the responsible alternative to OpenAI. That positioning is commercially valuable. It is also fragile. Every executive who echoes Coxson’s assessment — as two already have — makes it harder to maintain the current trajectory without acknowledging the gap between rhetoric and action.

Coxson’s call for government intervention and industry-wide coordination is not new. What is new is the clarity with which an insider has described why those mechanisms have failed so far. The problem is not a lack of warnings. It is that warnings, however credible, do not move markets or regulation without political pressure.

The next few months will tell whether Coxson’s departure triggers anything beyond an interview cycle — or whether it becomes another data point in a field that already has plenty of them. But the deeper measure will come later. It will be visible in whether the researchers who stay at Anthropic feel they still have a path to influence the company’s direction, or whether they begin, like Coxson, to calculate the cost of silence. That is the question this story is really about. Not one resignation, but what happens when the people best positioned to prevent disaster conclude that the structure around them makes prevention impossible.