OpenAI Is Now a Chip Company. Nvidia Should Be Worried

OpenAI Just Became a Silicon Company

The news dropped on a Tuesday, and most of the internet scrolled past it. OpenAI published test results for Jalapeño, its first custom-built inference chip, and the numbers were immediately striking. Against Nvidia’s GB200 and GB300 systems—the very hardware that has powered the AI boom—Jalapeño delivered 1.5 to 1.9 times more AI work per kilowatt of electricity, with end-to-end response latency 1.7 to 3.6 times lower.

This is not a marginal improvement. This is a company that built its entire fortune on a foundation it now intends to replace. OpenAI is no longer just a model company. It is a chip company. And that changes the math for everyone else in the room.

What the Numbers Actually Mean

OpenAI ran its benchmarks through SemiAnalysis’s InferenceX framework, testing against three open-source models: GPT-OSS 120B, DeepSeek R1 670B, and the largest, Kimi K2.5 at 1 trillion parameters. The pattern was consistent across all three.

With GPT-OSS, Jalapeño pushed 85,448 tokens per second per kilowatt, nearly double the 44,960 tokens the GB200 system managed. Response time dropped from 1.80 seconds to 1.03 seconds. With the even larger DeepSeek and Kimi models, the efficiency gap widened further. Per-user decoding throughput—tokens generated per second per customer during interactive chat—reached 700 on Jalapeño versus 169 on the Nvidia reference system. A 4.1x difference in the metric that actually matters to anyone using ChatGPT.

The distinction OpenAI keeps making is between compute and inference. Training remains an expensive beast where GPUs still hold advantage. But inference is where the money bleeds for companies like OpenAI. Every token served to ChatGPT is a cost line. Power bills, cooling, data center space—all of it scales linearly with demand. An inference chip that halves your energy cost per request is not a nice-to-have. It is an existential advantage.

The Architecture Behind the Performance

Jalapeño is not simply a faster GPU. OpenAI designed it as a complete system—processor, memory, networking, and software stack—all co-engineered. That choice matters because inference has two fundamentally different bottlenecks, and general-purpose chips treat them as the same problem.

The prefill stage, where the model processes the initial prompt, is compute-heavy. The decode stage, where the model streams tokens one by one into your screen, is memory-bandwidth-bound. Nvidia’s architectures are optimized for training, which skews heavily toward compute. OpenAI’s Jalapeño shifts the balance toward the decode path, keeping KV cache—the working memory that holds intermediate token states—close to where it is actually needed and cutting down data movement between processors.

The design sprint itself was notable. OpenAI claims it went from initial architecture to tape-out in nine months, using AI tools to accelerate the design exploration, verification, and optimization. They even deployed their own Codex and GPT-Astra systems during the process, achieving 1.5 to 1.8 times faster execution on certain attention and MoE blocks than the original hand-written implementations. The company is using its own AI to build the chip that will run its own AI. That loop is not coincidental.

The Implications for Nvidia

Nvidia has enjoyed something close to a monopoly on AI inference for over two years. Every major cloud provider, every startup, every research lab buys H100s and H200s, then B100s and B200s. The pricing power that delivers has made Nvidia one of the most valuable companies on earth. A single competitor building a chip that measurably outperforms that stack—at the exact point where inference economics matter most—is a crack in the armor.

The crack may widen. OpenAI confirmed that a second generation of Jalapeño is already significantly advanced, with a third generation in active design. That kind of iteration cadence suggests OpenAI does not see this as a one-off experiment. It sees it as a product roadmap.

Nvidia is not defenseless. The company continues to ship impressive hardware, and OpenAI explicitly stated it will keep using Nvidia accelerators for both training and inference. But the message is unmistakable: OpenAI is diversifying its supply chain, and it is doing so with hardware that is purpose-built for the workload that consumes the most electricity and the most money.

The Bigger Picture

This story has been building since 2023, when Anthropic began quietly recruiting TPU talent from Google and when Amazon started shipping its Graviton chips for inference. The trend is clear: the companies running the largest AI workloads are concluding that buying someone else’s chip is a strategic liability.

OpenAI’s move is the most consequential yet because of its scale. This is not a niche player experimenting with custom silicon. This is the company behind ChatGPT, the most widely used AI application on the planet, committing to the infrastructure that runs it. If Jalapeño delivers on its promise at production scale, it becomes a template. Every major model developer will face the same question: why rent inference capacity when you can build something cheaper, faster, and entirely under your control?

The deployment timeline is imminent. OpenAI plans to begin placing Jalapeño on its own infrastructure late this year, after finishing production qualification and software hardening. A 2027 public listing has also been floated, which would mean this hardware strategy becomes part of a much larger financial narrative soon.

Meanwhile, the competitive dynamics in the market continue to shift. DeepSeek and Kimi, the very models OpenAI used in its benchmarks, represent the kind of open-source pressure that makes proprietary hardware advantages even more valuable. OpenAI is not just building a chip. It is building a moat around the most expensive part of its business.

What happens next depends on whether Nvidia can respond quickly and whether other model developers follow OpenAI into custom silicon. But the direction is clear. The era of running the world’s most important AI on someone else’s hardware is ending, and OpenAI just turned off the lights.