FP32

FP32, or single precision, is a 32-bit floating point format with 1 sign bit, 8 exponent bits, and 23 mantissa bits. It stores each value in 4 bytes and has been the default numeric type for GPU compute, covering both graphics and general purpose calculation.

FP32 spans roughly 1.2 times ten to the negative 38th to 3.4 times ten to the 38th, with about 7 decimal digits of precision. A 70 billion parameter model stored in FP32 needs about 280 GB of memory, twice FP16 or BF16 and four times FP8. Most CUDA cores are built around FP32 throughput, and it remains the reference precision that lower precision formats get checked against for accuracy loss.

When a spec sheet lists a single TFLOPS number without qualification, it is usually FP32. Treat it as the baseline for comparing cards, then check separately whether the workload can drop to FP16, BF16, or FP8 for real speed and memory gains.

Sources

Last verified August 29, 2026.