The industry is moving away from FP32, FP16, and BF16 toward FP8 for machine learning inference and training. In cloud and data center systems, the smaller format increases throughput by allowing more computation per cycle on the same hardware. On edge and mobile devices, the reduced model size lowers memory footprint and bandwidth, which directly affects power consumption and latency.
Arm, Intel, and NVIDIA have published an OCPOpen Compute Projectan initiative for the standardization and open design of data center hardware 8-bit Floating Point Specification, OFP8, as an interchange format for sharing models across software systems. FP8 is already in use today, with support in NVIDIA's Hopper H100 GPUgraphics processing unitA GPU is a parallel graphics processor widely used for AI acceleration. and deployment in major ML frameworks. Arm has added FP8 support to the Arm v9 ISA1. Instrumentation Society of America2. instruction set architectureAn ISA defines the machine-level instructions a processor supports. and is working on how to integrate it across ML platforms.
FP8 is not a single encoding but a family of formats. The two proposed encodings are E4M3 and E5M2. With only 8 bits available, the formats make different trade-offs between precision and dynamic range, so software can choose the better fit for each tensor role.
E4M3 uses 1 sign bit, 4 exponent bits, and 3 mantissa bits, with a bias of 7. It uses most bit patterns for normal finite numbers rather than reserving many for special values, which helps extend dynamic range. E5M2 uses 1 sign bit, 5 exponent bits, and 2 mantissa bits, with a bias of 15. It follows IEEEInstitute of Electrical and Electronics Engineers 754 conventions more closely, including infinities and NaNs, and can be viewed as a lower-mantissa version of IEEEInstitute of Electrical and Electronics Engineers FP16.
The choice between the two encodings depends on tensor behavior. Activations and weights often have more stable value distributions after training or calibration and can benefit from more mantissa bits. Gradients can vary more strongly during forward and backward passes and therefore benefit from the larger dynamic range of E5M2. The format family allows these roles to be handled differently within the same workflow.
Arm reflects that idea in the Floating-point Mode Register, FPMR. The register is separate from FPCR and lets hardware track the format of each source operand, the destination format for conversion instructions, and scaling factors such as LSCALE and NSCALE. That design makes frequent per-tensor FP8 mode changes cheaper than pipeline-serializing control updates.
Arm's FP8 instruction support spans Advanced SIMD, SVE, and SME1. subject matter expert2. software maintenance engineerSoftware-related method or tool for development, control or data processing.3. semiconductor manufacturing equipment. The architecture extensions include conversion, scaling, widening multiply-accumulate, dot-product, and matrix-multiply instructions, with accumulation into wider types such as FP16 or FP32. This preserves the bandwidth benefit of FP8 inputs while keeping numerical stability during accumulation.