FP8

FP8 is an 8-bit floating point format that packs a sign bit, exponent bits, and mantissa bits into a single byte. Two layouts are in common use: E4M3, with 4 exponent bits and 3 mantissa bits, and E5M2, with 5 exponent bits and 2 mantissa bits. E4M3 trades range for precision, E5M2 trades precision for range, and a framework can pick between them tensor by tensor.

NVIDIA added FP8 support with the Hopper architecture's Transformer Engine in the H100, and later parts including Blackwell carry it forward. E4M3 covers roughly plus or minus 448, while E5M2 stretches to roughly plus or minus 57,344 with fewer significant digits. Storing a 70 billion parameter model in FP8 takes about 70 GB, versus 140 GB in FP16 or BF16 and 280 GB in FP32.

FP8 support is a hardware feature, not a given. Confirm the GPU generation includes FP8 tensor core paths before assuming a checkpoint will run at that precision, since pre-Hopper cards fall back to FP16 or INT8 instead.

Sources

Last verified August 29, 2026.