INT8

INT8 is an 8-bit signed integer format that stores whole numbers from -128 to 127. Quantized neural networks use it by dividing each floating point value by a scale factor, rounding, and clipping to that range. Dequantizing multiplies the integer back by the scale. At one byte per value, INT8 is half the size of FP16.

INT8 spaces its 256 values evenly, while FP8 in the E4M3 layout uses a floating point grid that reaches plus or minus 448. Intel's 4th Gen Xeon Scalable cores run 2,048 INT8 operations per cycle through Advanced Matrix Extensions (AMX). The prior generation managed 256 per cycle with AVX-512 Vector Neural Network Instructions (VNNI). NVIDIA's newest GPUs cut INT8 instead. An HGX B200 baseboard is rated 72 POPS (peta operations per second) of INT8 and 72 PFLOPS of FP8, both with sparsity. The Blackwell Ultra HGX B300 lists 3 POPS of INT8 beside the same 72 PFLOPS of FP8. A Rubin GPU lists 250 TOPS (trillion operations per second) of dense INT8 against 17.5 PFLOPS of dense FP8.

For CPU inference on Xeon processors with AMX or VNNI, quantize to INT8. On Blackwell Ultra and Rubin GPUs, quantize to FP8 or FP4 instead, since their INT8 throughput is a small fraction of FP8.

Sources

Last verified October 7, 2026.