Quantization and Low-Precision Number Formats: A Reference
A reference for the zoo of small formats LLMs now run on, typeset as a 17-page PDF.
Bit layouts, scaling and outliers, model quality, and which hardware runs what: FP16 and BF16 down to FP8, FP4, MX and NVFP4, plus the integer and lookup-table quant schemes (GGUF, GPTQ, AWQ, NF4, EXL2).