HBM

HBM, short for high bandwidth memory, stacks multiple DRAM dies vertically beside the processor. It connects through a silicon interposer instead of a normal circuit board trace. That short, very wide path gives it far more bandwidth per watt than GDDR or DDR memory sitting off to the side.

The HBM3 spec allows about 819 GB/s per stack, roughly 6.4 Gb/s per pin across a 1,024 bit interface. Real parts run below the spec ceiling. NVIDIA's H100 SXM carries 80 GB across five active stacks on a 5,120 bit bus, for 3.35 TB/s. Compare that to a high-end GDDR6X card at around 1 TB/s, or DDR5 system memory at roughly 40 to 50 GB/s per channel.

That packaging is why accelerator memory is fixed at the factory and cannot be upgraded or swapped later. It also accounts for a large share of an accelerator's bill of materials. More stacks and more active die area cost real money, and yield on the memory stack matters as much as yield on the compute die.

Sources

Last verified August 29, 2026.