FP4

FP4 is a 4-bit floating point number format used to store neural network weights and activations at very low precision. It typically follows an E2M1 layout, 1 sign bit, 2 exponent bits, and 1 mantissa bit, so a single value can represent only a handful of distinct magnitudes. NVIDIA's NVFP4 and the open MXFP4 format restore usable accuracy by grouping values into a block and sharing one higher precision scale factor across it.

NVFP4 uses blocks of 16 values with an 8-bit scale factor, working out to about 4.5 bits per value including overhead. MXFP4 uses blocks of 32 with an 8-bit shared exponent, or 4.25 bits per value. A single B200 GPU delivers 18 PFLOPS of dense FP4 tensor throughput, double its 9 PFLOPS at FP8. A model quantized to FP4 needs about half the GPU memory it does at FP8. A 70 billion parameter model drops from roughly 70 GB of weights to roughly 35 GB.

FP4 support is a Blackwell generation feature. Hopper and earlier GPUs lack the tensor core hardware to run it natively, so an FP4 checkpoint has to be converted to a higher precision before it runs. Halving weight memory changes how many GPUs an inference deployment needs.

Sources

Last verified August 29, 2026.