OpenAI Jalapeño Chip Results: Benchmarks, Efficiency and Rollout

What Jalapeño is

Jalapeño is OpenAI’s first custom silicon, developed in close collaboration with Broadcom and first announced the previous October. It serves inference only; it does not train models. OpenAI presented the first benchmark results at the Hot Chips conference on August 25, 2026, alongside a written account of how they were produced. [1][3]

The design starts from the observation that language-model serving has two phases with different bottlenecks. Prefill, which processes the prompt, is compute-bound; decode, which emits tokens one at a time, is bound by memory bandwidth. A system tuned for one can lose its advantage waiting on data in the other. Jalapeño keeps model state, including the KV cache, placed locally and uses a network domain large enough to hold a whole request inside one connected system, so the phases do not pay for moving state between resources. [1]

What was measured

OpenAI ran InferenceX, a public benchmark from SemiAnalysis that measures the full path of serving a request, against three open-weight models: GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. Across all three it reports 1.5 to 1.9 times more work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems, rising to 2.1 to 4.1 times higher performance on highly interactive workloads. [1]

The comparison systems are named in the appendix: an NVIDIA GB200 for GPT-OSS and a GB300 for the two larger models. Results are normalized by each accelerator’s published package power rating, which is 700 watts for Jalapeño against 1,200 and 1,400 watts. OpenAI notes that Jalapeño’s measured sustained power stayed at or below 550 watts on the workloads tested. [1]

Atlas interpretation: Normalizing by rated rather than measured power is a defensible choice and it cuts in an unobvious direction. It understates Jalapeño, which drew less than its rating, and it equally credits the comparison parts with their full nameplate whether or not they drew it. So the per-watt ratios are not measurements of energy consumed; they are throughput per unit of provisioned power, which is the number a data center actually plans against but not the number a power bill reflects. [1]

How to read the very large multiples

The appendix carries much larger figures than the headline: roughly 53.7 times more throughput per kilowatt on GPT-OSS, 104.3 times on DeepSeek R1 and 56.1 times on Kimi K2.5. Each is labelled "more throughput at previous TBT," meaning throughput measured at the fastest per-user token rate the comparison system could reach at all. [1]

Atlas interpretation: Those numbers describe the edge of the other system’s operating range, not its normal one. At the point where a GB300 can just barely sustain 169 tokens per second per user, it has almost no throughput left; Jalapeño at the same latency still has plenty. That is a real and useful property for interactive agents, which pay for latency on every step of a chain. It is not a claim that the chip is a hundred times faster, and the honest headline numbers are the 1.5 to 1.9 and 1.7 to 3.6 ranges. [1]

Atlas interpretation: The other limit is who ran it. InferenceX is a public harness, which is better than a private one, but OpenAI configured and ran both sides and is reporting on its own chip. OpenAI also says its advantage widened further on its own frontier models in internal testing, which is a claim nobody outside can check at all. [1][3]

The chip was designed with models, and for them

OpenAI says its own models moved the project from initial design to tapeout in nine months by exploring implementations, shortening verification loops and optimizing arithmetic circuits. The chip was also deliberately made an easy target to program: work is described through local tensors, explicit communication and predictable synchronization, so a model has a tractable structure to map work onto. [1]

The concrete result is a porting figure. Using Codex with GPT-Astra, the team brought three open-weight models that were never in Jalapeño’s production plan to high performance within two months. For selected GPT-OSS attention and mixture-of-experts blocks, model-generated implementations ran 1.5 to 1.8 times faster than the existing expert-written ones. OpenAI states that this applies to selected blocks and not to the full model. [1]

Atlas interpretation: Kernel work is the tax that has historically kept custom accelerators from being usable outside their designer’s own workloads: every new model family needs new kernels, written by scarce people. If that tax falls, the argument for buying general-purpose hardware because the software ecosystem is there gets weaker. That is a bigger claim than any of the benchmark ratios, and the evidence for it is currently three ported models and some hand-picked blocks. [1]

What it changes, and when

OpenAI plans to begin deploying Jalapeño inside its own infrastructure by the end of 2026. Its head of hardware put that first deployment at "very small volumes," with more significant deployment in 2027. Gen 2 is described as deep in development and Gen 3 as taking shape. OpenAI also says it will continue to widely deploy NVIDIA accelerators for training and inference. [1][3]

Atlas interpretation: Two things bound the result. Jalapeño is not for sale, so it does not take a socket away from anyone; it reduces how many OpenAI has to rent or buy. And by the time it ships in volume, the comparison is no longer a GB200 or GB300. A chip beating the currently shipping part on a benchmark run a year before its own volume deployment is evidence about design capability, not about a market share number. [3]