model quantization

Reducing a trained model's numeric precision (e.g. 32-bit floats to 8-bit integers) to shrink its memory and latency footprint.

Quantization reduces a trained model's weights and/or activations from a higher-precision numeric format (typically 32-bit floating point) to a lower-precision one (commonly 8-bit integers), shrinking the model's memory footprint and often its inference latency, at a usually small but nonzero cost in accuracy.

Examples

Quantizing a 400MB float32 model to 8-bit integers shrinks it to roughly 100MB and speeds up inference, at a small accuracy cost worth re-measuring on a held-out set.