Why OpenAI is building its own computer chips

OpenAI has announced that its new chip, developed in partnership with the hardware company Broadcom, significantly outperforms current high-end industry chips in specific tests. In these benchmarks, the chip showed it could generate more text per user and handle more work for every unit of electricity it consumes. OpenAI describes the chip as being purpose-built for inference, which is the stage where a trained AI model is put to work answering questions or completing tasks. The company plans to start using these chips in small numbers by the end of 2026, with a wider rollout planned for 2027.

To understand why a custom chip matters, think of an AI model like a chef in a kitchen. Training an AI is like teaching the chef every recipe in the world, which requires a massive, stationary kitchen. Inference is the actual act of the chef cooking a meal for a customer. Currently, most AI services run on general-purpose chips that were not originally designed solely for this kind of cooking. This leads to bottlenecks, much like a kitchen where the pantry is too far from the stove. Every time the chef needs an ingredient—or in the computer's case, a piece of data—it takes time to travel back and forth. Jalapeño is designed to be the ultimate custom-built kitchen for AI. It keeps the essential data, such as the AI's internal state and short-term memory, stored right next to the processing parts of the chip. By minimizing the distance data has to travel, the system can whip up responses much faster and with less wasted energy, which is why it is both speedier and more efficient than existing solutions.

This move signals that for companies like OpenAI, the limiting factor for AI growth is no longer just the intelligence of the models themselves, but the physical hardware they run on. As more people use AI, the demand for electricity and processing power becomes a major hurdle. By building its own chips, OpenAI can optimize the hardware specifically for the way its own software behaves, rather than relying on a one-size-fits-all product. While this doesn't mean OpenAI will stop working with traditional hardware partners like Nvidia, it highlights a broader industry shift where the most powerful AI companies are becoming hardware companies as well. For the user, this points toward a future where AI assistants become more responsive and reliable, even as millions of people use them at the exact same time.