Japan Found OpenAI's Strongest AI Flaw — And Western Outlets Missed It

The report nobody in the West saw coming

On September 7th, gamespark.jp published a report about something far more interesting than its headline suggests. OpenAI’s latest model, GPT-6 Astra — widely treated as a stepping stone toward artificial general intelligence — can beat Valve’s 2007 puzzle game Portal in under two hours of game time. The real number, though, is twenty-three hours and thirty-eight minutes. That gap tells the whole story.

GitHub user cozyblaze documented the experiment. The method is straightforward: run the game for a few seconds, pause, capture a screenshot, feed the image to GPT-6 Astra, let the model think while the game stays paused, then execute whatever camera movement and button press the model decides on. Repeat. Six thousand nine hundred twenty-five operations. Five hundred seventy-one dollars in token costs.

The model finished. But how it finished matters more.

The physics problem

A YouTube video of the run shows GPT-6 Astra using the portal gun with reasonable competence. It places portals on walls and floors, navigates rooms, solves the kind of spatial logic puzzles Portal is designed to teach. Where it stumbles — and this is the part that should alarm anyone tracking frontier AI capability — is with momentum.

Moving platforms. Velocity-dependent puzzles. Situations where the correct move depends on understanding how far you’ll travel after leaving a surface. The model hesitates, misjudges, sometimes walks off edges it should have avoided.

This is not a minor failure mode. It is a structural one.

GPT-6 Astra processes individual frames. It does not maintain a continuous internal model of motion through time. Each decision is a discrete classification task dressed in reasoning language. The model can understand that a platform moves, but it cannot easily simulate forward what will happen when it steps onto one and carries momentum across a gap. That is not a data problem. It is an architecture problem.

The implications extend far beyond a puzzle game. Portal’s momentum-based design forces players to internalize physics they cannot see — calculating trajectory, estimating velocity transfer, trusting that an action taken now will produce a specific outcome later. A human player develops this intuition through embodied practice. The model has none of that. It sees a frame. It predicts a move. It does not carry forward a coherent model of the world from one moment to the next.

Researchers who have studied AI alignment and robustness have long warned that pattern-matching systems fail in predictable ways when pushed into novel situations. The Portal experiment makes this visible in real time. Watch the model enter a chamber with a moving platform and watch it treat the problem as static. The platform moves. The model does not adjust. It fires a portal based on where the platform was, not where it will be. This is not a bug. It is the natural output of a system that reasons in snapshots.

Why Japan got there first

Western outlets have been covering GPT-6 Astra’s releases in broad strokes — benchmark scores, AGI proximity headlines, the usual circuit. No major English-language publication appears to have covered the Portal experiment. Gamespark, a site built around Japanese gaming news, did.

The writer, K.K., is a psychology graduate who has been covering games for the site since 2022. His bio mentions a fondness for sci-fi, open-world games, and military fiction. He also admits he is dense to other people’s romantic feelings but understands animal emotions better — then blames it on not having a tail or ears. The piece itself is factual and measured, but it is filed under a section about AI capabilities in gaming, which is exactly where this story belongs.

The gap in coverage is not about quality. It is about attention. Western tech media is chasing the AGI narrative with a ferocity that leaves little room for the kind of granular, unglamorous testing that actually reveals what these models can and cannot do. A Japanese gaming outlet running a Portal speedrun with an AI agent is the opposite of a headline. It is a footnote. Which is why it got written.

There is also a structural reason worth noting. Japan has a long tradition of treating video games as serious cultural objects worth rigorous analysis. Games journalism there tends to prioritize craft, mechanics, and player experience over hype cycles. When a gaming outlet tests an AI against a puzzle game, it approaches the question from a different angle than a tech blog approaching it from the angle of market disruption or capability milestones. The Japanese coverage focused on how the AI played, not whether it represented a breakthrough. That difference in framing is itself a kind of signal.

The cost of a thought

$571 to complete Portal is not trivial. It is also not surprising if you understand the mechanics. Each of the 6,925 operations requires an image input and a text output. The model pauses the game while it thinks. In a game where real-time execution would let a skilled player solve the final chamber in under three minutes, GPT-6 Astra spent roughly twenty-two hours in pure decision-making time.

Most of those decisions were correct. Most were not fast. The model is trading compute for reasoning in a way that human players do not need to. A human sees a moving platform and knows, without calculation, whether they can make the jump. The model sees a static frame, runs it through a reasoning pipeline, and produces an action. The difference is not intelligence. It is embodiment.

The financial cost raises a practical question that the coverage has mostly ignored: at what point does this approach become viable for anything beyond novelty experiments? $571 for Portal is expensive. Scale that to a more complex environment — a game requiring hundreds of hours of interaction, multiple physical systems, dynamic environmental changes — and the token costs become prohibitive. Even GPT-6 Astra cannot sustain this strategy indefinitely. The experiment is impressive precisely because it is uneconomical.

Second-order effects

The Portal experiment will ripple through several communities in ways that are already becoming visible. Speedrunning communities, which have spent decades optimizing Portal completion through glitch exploitation and frame-perfect inputs, are treating the AI run with a mix of fascination and disdain. Some see it as a new category of play. Others see it as a non-player competing in a space designed for human expression. The tension between these views is productive. It forces a conversation about what counting as achievement in interactive systems.

AI safety researchers are taking note for a different reason. The failure modes observed here — momentum misjudgment, lack of temporal continuity, overreliance on static reasoning — are the same classes of error that appear when these models are deployed in robotics, autonomous systems, and medical diagnostics. A model that cannot reliably track changing physics in a simple game is unlikely to handle the continuous uncertainty of the real world. The Portal experiment is small. The warning it carries is not.

There is also an economic angle. The $571 cost per run creates pressure on the benchmark industrialization that OpenAI and other labs have built. If the most credible tests of reasoning require thousands of discrete inference calls per minute, the cost of validation scales non-linearly. This could incentivize cheaper but less honest benchmarks — the kind that measure superficial pattern matching rather than genuine reasoning. The Portal experiment, by being expensive and slow, inadvertently highlights the fragility of the current evaluation ecosystem.

What happens next

cozyblaze has said they want to try the same experiment with Portal 2. That is worth watching. Portal 2 introduces more complex momentum puzzles, environmental hazards that require timing, and a narrative structure that demands memory across chapters. If GPT-6 Astra struggles with basic moving platforms, Portal 2 will expose the limitations more harshly. The sequel also adds gels that alter physics properties, portal-compatible surfaces that change based on player placement, and turrets that require tactical reasoning about line of sight and reload cycles. Each new mechanic is another stress test for a system that already falters on simplicity.

More broadly, this experiment reveals something the AGI conversation keeps ignoring. Passing a benchmark — even a cleverly constructed one — does not mean a model understands the world. It means it has learned to map visual inputs to action outputs in a specific context. The model solved Portal. It did not solve it the way a human does. It solved it the way a very expensive calculator does: frame by frame, pause by pause, dollar by dollar.

The real story here is not that GPT-6 Astra can play Portal. It is that the Western press missed the report entirely, and that the failure modes embedded in the success are the exact ones that matter most for anyone tracking whether these systems are approaching general reasoning or simply getting better at pattern matching in confined environments.

Japan already knew. The question is whether the rest of the world will catch up — or whether the next experiment that matters will again go uncovered while everyone watches the wrong highlight reel.