The AI Safety Whistleblower Moment Has Finally Come

The Leak From Anthropic Is Not Just a Story — It’s a Signal

Evan Hubinger did not publish a peer-reviewed paper. He posted on X. In under a minute, the message reached more than ten million people: a senior researcher at Anthropic believed there was more than a ten percent chance that artificial intelligence could kill all humans within the next decade.

Ten percent. That may sound abstract — a number you can shrug off or fold into a risk matrix — but it is meant to feel concrete. It is the kind of probability that a doctor would not let a patient walk out of a clinic. And Hubinger, who works on AI alignment, said the risk from models that exist today is low. What worries him is self-improvement: AI systems that rewrite their own code fast enough to outrun the humans who built them.

That the post came the same week as Jacob Coxon — a former OpenAI researcher who had also just left Anthropic — declaring that neither company was acting responsibly gives the exchange a shape that looks less like coincidence and more like a coordinated signal from inside the industry.

Neither man specified a mechanism for human extinction. That omission matters. For years, the public debate about AI risk lived in philosophy departments and speculative essay contests. Now it is being stated in plain language by people whose paychecks depend on the very companies they are warning about.

Who Wins and Who Loses

The immediate winners are researchers who have spent years telling the same story to indifferent audiences. Hubinger and Coxon now have a platform that no academic journal can match. Their words will be quoted in Parliament, in congressional hearings, in op-eds, in policy drafts.

The losers are harder to name because the damage has not yet been measured, but the direction is visible. Any company that wanted to position itself as simply building useful chatbots and coding assistants now has to answer a question it did not have to answer last month: why should we trust you with infrastructure when your own researchers say you do not know how to keep a superhuman system aligned with human values?

Anthropic’s public posture has always been defined by caution. The company’s founding narrative, its emphasis on constitutional AI, its decision to release Claude in stages rather than all at once — these were marketed as evidence of responsibility. The posts from two of its researchers suggest that the marketing may be outpacing the engineering.

Dame Wendy Hall, who advises the United Nations on AI, was blunt. She told the BBC she was shocked. She also raised a possibility that is uncomfortable for everyone involved: some of this could be performance. Anthropic and OpenAI are both preparing for anticipated stock market debuts. A researcher publicly warning about existential risk while the company prepares to go public looks, to some observers, like a calculated move. Whether it is or is not, the mere suggestion is toxic to the credibility of the warning itself — and that is the second-order harm of Hall’s observation.

The Policy Door Opens

The most consequential development in this story is not on X. It is a letter from Darren Jones, the former chief secretary to the Treasury under Sir Keir Starmer, addressed to Prime Minister Andy Burnham. Jones called for a new multinational treaty governing the safe development of AI. His argument was simple and alarming: if governments do not step up now, the pace of development will outrun the pace of governance and the problems will arrive before anyone has begun to study them.

Jones is not a fringe voice. He sits at the center of British economic policy. His intervention is a bridge between the lab and the legislature, and it is exactly the kind of bridge that the AI safety movement has been trying to build for years.

Complicating matters further, the Financial Times reported that Anthropic withheld its latest model from the UK’s AI Security Institute — one of the world’s most prominent bodies for assessing AI risk. Anthropic declined to comment. The Cabinet Office declined to confirm or deny, offering only that it continues to collaborate with industry partners to make models safer. That non-denial is a data point in itself. It suggests either that the withholding is real and contested, or that the government does not want to escalate the dispute publicly. Either way, a company that built its brand on safety is being asked to explain why it would not submit its product to the very institution designed to evaluate that claim.

The Alignment Problem Is Still Unsolved

Hubinger did not mince words about alignment — the effort to encode human ethical principles into AI systems so they do not drift toward goals humans do not share. He said Anthropic does not yet have a plan to solve it for superintelligence and is not clearly on track to develop one. That admission from inside the company carries more weight than any external critique.

Anthropic’s own August safety report, meanwhile, acknowledged a shift in confidence. The company had previously assessed the risk of its models being misaligned with a powerful organization’s desires and then exploiting or tampering with systems as low. It also rated the risk of highly capable AI performing automated research that could cause catastrophic harm as low. But the report added a qualifier: it was less confident in that assessment now than it had been before.

This is the language of an organization that has seen new data and does not like what it says. The report also noted early signs of potential acceleration. “Acceleration” in AI safety literature is not a metaphor. It refers to the possibility that improvements in one capability feed improvements in another, creating a feedback loop that outstrips human oversight.

OpenAI faced a parallel reckoning earlier this summer, when its chief scientist, Jakub Pachocki, called for extreme caution and warned that more intervention may be needed to ensure humans remain in control of the future. Across the industry, major figures including Anthropic’s own Dario Amodei and Jared Kaplan have recently urged a slowdown in frontier development. A letter signed by 1,300 AI workers called on the US government to support an international effort to pace the frontier deliberately.

The pattern is clear: the people closest to the technology are the ones most alarmed by it. The people with power over its deployment are not in the same room.

What Happens Next

The next few months will determine whether these warnings produce policy or performative outrage. A multinational treaty is a tall order. Even the Paris climate agreement, which had decades of preparation, took years to negotiate. AI moves faster than diplomacy. But the alternative — waiting until a crisis forces a response — is what Hubinger and Coxon are warning against.

The stock market angle cannot be ignored. If Anthropic and OpenAI both go public in the near term, investors will face a choice: bet on a company whose own researchers say the technology may be uncontrolled, or wait. That tension will play out in prospectuses, in earnings calls, in shareholder resolutions. It will also play out in regulatory filings, because governments that take Jones’s letter seriously will begin asking questions that no investor relations team can answer with a slide deck.

For now, the balance of power remains with the labs. But the leak has changed the arithmetic. The debate is no longer confined to laboratories and conference halls. It is in the columns of newspapers, in the offices of treasuries, in the portfolios of pension funds. That is the real shift — and it is only the beginning.