OpenAI's Math Release Breaks the Benchmark Signal

The benchmark paradox

OpenAI did something unusual on Tuesday night. Rather than keeping its latest math results locked inside proprietary research, it published 377 solutions spanning algebra, number theory, theoretical computer science, mathematical logic, and topology. Ten of those solutions included summaries of how the model arrived at each answer, generated over roughly three hours of computing per result.

The move came two weeks after the company claimed to have solved the Navier-Stokes equation—one of the seven Millennium Problems, each carrying a $1 million prize—and faced immediate pushback from mathematicians who questioned whether the result represented genuine reasoning or merely a final step tacked onto someone else’s work.

OpenAI framed the new release as responsive to those concerns. It cited the advice of an independent advisory board hosted by the Institute for Advanced Study in Princeton. The board, formed after the Navier-Stokes controversy, recommended that AI labs release both their prompts and the full chains of thought used to produce results. OpenAI pointed to that guidance in its announcement.

But the release also exposed a fault line that runs through the entire field: OpenAI does not agree with the board’s recommendation that companies should stop testing proprietary models against advanced mathematical problems. Dan Roberts, OpenAI’s research lead, called the testing essential for building better tools. The proofs, he said, were a byproduct.

Who loses when benchmarks break

The non-obvious consequence of this release is not that AI can now solve hard math. It is that the act of publishing these results publicly destroys the very signal the field has been using to measure whether AI is improving.

For years, the benchmarking game worked like this: researchers pose a hard problem, a company tests its model against it, and if the model fails, the benchmark remains intact as a measure of distance to goal. Solve it, and you announce the win. Move on to something harder.

OpenAI has now effectively solved dozens of problems that were being used as next-step benchmarks—the kinds of problems you would hand to a graduate student or a postdoc to test whether a system could push into genuine mathematical research territory. The company created these benchmarks after the earlier Olympiad-style problems became too easy. Now those are gone, too.

What replaces them? Nobody knows yet. OpenAI has the internal models still testing against problems it has not published. The rest of the field is left trying to calibrate progress on a moving target whose coordinates have just been dumped into the public record.

Tristan Buckmaster, a mathematician at New York University who was working on the Navier-Stokes problem, said the current release shows insufficient due diligence. He also raised a more unsettling possibility: that mathematicians who have been feeding ideas into AI systems—through papers, preprints, or even casual online discussions—may have inadvertently provided the raw material that allowed OpenAI to complete proofs before the humans who started them could.

There is likely to be a bunch of results where they take someone’s work and then take it to completion, Buckmaster said.

The speed vs. understanding problem

The deeper tension is not technical. It is epistemic. Mathematics advances when proofs are understood, internalized, and built upon—not when they are simply verified and filed away.

Many of the 377 proofs were checked using Lean, a formal language that verifies logical correctness. A Lean check confirms that the argument is valid. It does not confirm that a human mathematician can read the result and extract insight from it.

Kai Shaikh, a graduate student in mathematics at the University of Toronto, put it this way: the issue comes down to two competing human impulses—competition and the appreciation of beauty. To me this seems to be a case of the former attempting to strangle the latter, he said.

Melanie Wood, a mathematician at Harvard and member of the advisory board, called the release the beginning, not the completion, of the process of human understanding and the incorporation of the work into mathematical knowledge.

That framing matters. It acknowledges that verification and publication are not the same as comprehension. And it suggests the real work for mathematicians is just starting—not the AI’s.

What happens next

The immediate aftermath will likely involve a wave of mathematicians trying to reconstruct the provenance of each result—determining which open problems were close to being solved by human researchers and whether OpenAI’s models simply crossed the finish line first. Some of the 377 results may turn out to be incremental completions of partially written arguments. Others may represent genuinely novel approaches. The field will not know for months.

OpenAI’s internal testing continues against problems it has not released. That means the company’s true performance ceiling remains hidden behind a wall of unpublished benchmarks. The advisory board’s recommendation to stop testing on advanced mathematical problems has been rejected. The race is not slowing down.

What is slowing down is the ability of anyone outside OpenAI to measure where the race stands. The benchmark signal has been compromised. The field is now flying partly blind, trying to understand results that arrived faster than the community could process them.

The release was framed as transparency. In practice, it functioned as a calibration shock—proving capability while making it harder for the rest of the world to track what comes next.