Microsoft's Maia 300 Bets on Memory, Not Muscle, to Cut AI Costs

Why memory, not compute, is the headline

Microsoft didn't abandon raw performance; it decided the bottleneck had moved.

According to Microsoft's architecture deep dive, the Maia 300 moves from a single die to a multi-chiplet package: compute on TSMC's N2P process, with memory controllers and I/O split onto separate dies. Eight HBM4 stacks deliver 288GB at 18TB/s per accelerator, against 216GB at 7.1TB/s on Maia 200.

That is a 2.5x jump in bandwidth for a 36% gain in compute. It sounds lopsided until you look at what inference actually does. In decode, the token-by-token phase that dominates long reasoning chains and agent loops, accelerators spend much of their time waiting on data rather than doing arithmetic.

Microsoft's answer is to keep data close to the compute that uses it: a non-uniform memory layout across chiplets, software-managed on-die SRAM, and narrower number formats including an enhanced MXFP4 and sub-3-bit storage with hardware conversion between them.

The Nvidia math Microsoft is quietly attacking

Nvidia's own earnings show why every in-house inference chip matters.

In the quarter to July 2026, Nvidia reported $96.2 billion in revenue, $89 billion of it from data centers, at a 75% gross margin. CFO Colette Kress put hyperscale revenue at $49 billion, meaning a little over half of Nvidia's data center business comes from a small group of buyers, the biggest of which now design their own accelerators.

Every inference workload that moves in-house is one where that buyer pays closer to manufacturing cost instead of Nvidia's margin. Microsoft told investors in July that Maia 200 was up to 40% cheaper to operate than Nvidia's latest chips on OpenAI and Microsoft models. It did not name the GPU it compared against, which matters given Nvidia's efficiency push with Vera Rubin.

The market barely blinked. Nvidia shares closed down 1.29% after the reveal while Microsoft's rose 1.14%, and Evercore ISI analysts estimate Maia will account for only about 10% of Microsoft's AI chip spending by the first quarter of 2027, CNBC reported.

Everyone is going the Google route now

Custom silicon stopped being a side project for hyperscalers some time ago.

Two years ago Microsoft told the European Commission that Google was the only company vertically integrated at every AI layer, from chips to a mobile app store, and that everybody else had to rely on partnerships. The Maia 300 announcement is Microsoft taking that route itself.

Google now recognizes revenue from TPU systems sold into customer data centers, CFO Anat Ashkenazi said on Alphabet's July earnings call. Anthropic has lined up 3.5GW of TPU capacity through Broadcom for 2027 on top of an earlier deal for up to a million TPUs. Amazon CEO Andy Jassy said in April 2026 that its chip business, spanning Trainium, Graviton and Nitro, passed a $20 billion annual revenue run rate. OpenAI has shown its own Jalapeño accelerator.

None of that has stopped anyone from buying Nvidia. Amazon is still set to take more than 3 million Nvidia GPUs by mid-2028, and Alphabet raised its 2026 capex guidance to between $195 billion and $205 billion while listing Vera Rubin alongside its own TPUs. The pattern is clear: in-house chips handle the work you control, Nvidia handles the rest.

Availability and pricing

Microsoft has not announced retail pricing or a public launch date for Maia 300. The chip is an Azure accelerator, not a product you can buy, and the company says multi-gigawatt deployments begin in 2027. Access for developers will come through Azure AI services, with no published rates yet.

For US, UK and EU customers, the practical effect shows up in Azure inference pricing rather than in a spec sheet. If Microsoft's 40% operating-cost claim holds at scale, some of that gap could reach cloud bills, but Microsoft has not committed to passing savings on.

The Maia 300 is less a Nvidia killer than a negotiating tool. Microsoft gets a cheaper path for the inference it runs itself, while keeping Nvidia close for training and for customers who want it. That is a meaningful shift in leverage, and 2027 is when we find out how much.