The SRAM Question Hanging Over the Memory Trade: Does Inference Still Need HBM?
The memory bull case rests on a single assumption, and it is worth stating plainly because everything else follows from it: every incremental dollar of AI compute requires proportionally more high-bandwidth memory. GPUs pair with HBM, HBM is scarce and expensive, and that scarcity is precisely what handed Micron and SK Hynix gross margins near 85% and market caps north of a trillion dollars. If the assumption holds, memory demand scales with the AI buildout indefinitely. The SRAM wildcard is the possibility that a meaningful slice of AI demand quietly stops needing the memory these companies sell.
Why Inference Is Different
The distinction that matters is between training a model and running it. Training — teaching the model — is compute-heavy and arithmetic-intensive, and GPUs packed with HBM dominate it. Inference — actually answering a query — is a different animal. It is memory-bandwidth bound. To generate each token, the chip must read the entire model out of memory, do a small amount of math, and move on to the next token. Because autoregressive decoding can’t start the next token until the current one finishes, the whole process is gated by how fast you can move weights out of memory, not by how much math you can do.
That is exactly the problem SRAM solves. Static RAM sits directly on the compute die, so accessing it is physically faster than reaching out to external HBM stacks. Store the model weights in on-chip SRAM, right next to the cores, and you sidestep the memory wall that throttles GPU inference.
The Companies Betting Against HBM
Cerebras is the sharpest expression of this. It builds wafer-sized chips — dinner-plate-scale silicon — carrying 44 GB of on-chip SRAM and delivering roughly 21 petabytes per second of memory bandwidth, which the company translates into inference up to 15x faster than a top-end GPU. The pitch from CEO Andrew Feldman is not subtle: HBM is the binding constraint in the market, it is in short supply, it is expensive, and Cerebras simply does not use it. He makes the same argument about TSMC’s CoWoS advanced-packaging queue and leading-edge process nodes — all constraints the wafer-scale design sidesteps.
This is not a lone startup talking its book. Nvidia’s acquisition of Groq brought another SRAM-centric inference architecture in-house. OpenAI signed a Cerebras deal in January and is reportedly seeing sharply reduced inference costs as a result. When the largest buyers of AI compute start allocating inference to architectures that bypass HBM, the read-through to memory makers is direct: the HBM-content-per-token figure baked into their valuations may be too high.
Why It’s a Wildcard, Not a Thesis-Killer
Here is the other side, and it is substantial. SRAM has a hard physical ceiling: capacity. Even a full Cerebras wafer holds only 44 GB — a rounding error next to what large models need. To fit a one-trillion-parameter model you have to network roughly 45 CS-3 wafers together, which can cost $100 million or more. A single Nvidia GB200 rack holds several such models in HBM for around $3.5 million. On a pure capacity-per-dollar basis it isn’t close.
Then there is the KV cache — the memory that holds all the context from a user’s session. As AI agents run longer, multi-turn reasoning chains, that cache explodes in size, quickly overflowing SRAM and even HBM, forcing systems to lean on more DRAM and SSD behind the fast tier. This is why Nvidia has been adding SSD to its servers, not removing memory. SRAM-centric systems are Formula 1 cars: unbeatable low latency for a customer willing to pay up — a hedge fund’s trading desk, a latency-critical robotics application — but not the vehicle for mass-market chatbot APIs serving thousands of concurrent users, where sheer memory capacity wins.
The likeliest real-world outcome, then, is not substitution but stratification: a deeper memory hierarchy of SRAM plus HBM plus DRAM plus flash, each tier matched to the workload it serves best. Several analysts argue this expands the total memory market rather than cannibalizing it — more layers, not fewer. HBM demand is not weakening today. The nuance is that HBM may not be the universal bottleneck across every AI architecture, and the market may need to separate training-heavy GPU demand from fast-inference demand that runs on different memory systems.
Why It Still Moved the Stocks
If the balanced conclusion is “layered hierarchy, expanded TAM,” why did SRAM headlines contribute to a double-digit memory selloff this week? Because a de-risking market does not price the nuanced outcome. It prices the tail. When a stock is up 300% to 800% and valued for perfection, the relevant question is not “will HBM demand keep growing” — it obviously will — but “what if the single assumption underpinning this entire run is even slightly wrong?” That is enough to trigger selling in extended names, regardless of how the debate ultimately resolves.
The SRAM wildcard is best understood not as a prediction that HBM demand is falling, but as the market’s dawning recognition that the memory thesis has an untested joint: the belief that all AI compute is HBM-hungry in the same proportion. Cerebras, Groq, and OpenAI’s inference economics are the first real stress on that joint. It probably doesn’t break. But now that investors can see it, they will price it — and in a crowded, parabolic trade, simply making a hidden assumption visible is enough to move the tape.