Memory-bound vs compute-bound

A workload is memory-bound when moving data limits its speed, and compute-bound when the processor's math rate does. The deciding number is arithmetic intensity, the floating-point operations (FLOPs) performed per byte moved to or from memory. A processor's ops:byte ratio is its peak math rate divided by its memory bandwidth. In the roofline model of Williams, Waterman and Patterson, that ratio marks the ridge point, where the sloped memory roof meets the flat compute roof. Work below it is memory-bound, and work above it is compute-bound.

The RTX PRO 6000 Blackwell Workstation Edition peaks at 503.8 TFLOPS of dense 16-bit floating point (FP16) tensor math. With 1,792 GB/s of memory bandwidth, its ridge point sits near 281 FLOPs per byte. NVIDIA's guide puts one linear layer at 1 FLOP per byte at batch size 1, far below that ridge. At batch size 512 the same layer reaches 315 FLOPs per byte, above it. Single-user large language model (LLM) decode is the batch-size-1 case. Prefill and training push many tokens through the same weights at once.

Work out a workload's FLOPs per byte before choosing hardware. Below the GPU's ratio, buy memory bandwidth. Above it, buy TFLOPS. For single-user chat decode, rank cards by GB/s.

Sources

Last verified October 7, 2026.