Curriculum

Edge AI · Hardware Optimisation

Model Quantisation

Quantisation reduces model weight precision from 32-bit floats to 8-bit or 4-bit integers, cutting memory footprint 4–8× and accelerating inference on integer-optimised hardware.

The idea

Neural network weights and activations are typically float32 (4 bytes per value). A 7B parameter model requires 28 GB in FP32. Quantising to INT8 (1 byte) reduces this to 7 GB; INT4 to 3.5 GB—fitting on a consumer GPU.

Post-Training Quantisation (PTQ): quantise a pre-trained FP32 model without retraining. Calibrate the quantisation ranges using a small dataset. Fast but may degrade accuracy, especially at INT4. GPTQ and AWQ are popular PTQ algorithms for LLMs.

Quantisation-Aware Training (QAT): simulate quantisation during training using fake-quantisation operators. The model learns to be robust to quantisation error, recovering most or all accuracy. More expensive than PTQ but produces higher-quality INT8 models. Standard for edge deployment of CNNs in mobile vision apps.

x_q = round(x / s) + z x ≈ s·(x_q − z)

Affine quantisation: s is the scale factor, z is the zero-point. The quantised value x_q is an integer.

What to hold onto

  • Mixed-PrecisionKeep sensitive layers (first/last, attention) in higher precision; quantise the rest to INT4. AutoGPTQ and llama.cpp implement this heuristically.
  • Activation QuantisationQuantising activations (not just weights) is harder due to dynamic range variability. Smooth-Quant shifts quantisation difficulty from activations to weights via channel-wise scaling.
  • Hardware AccelerationINT8 GEMM on NVIDIA Tensor Cores runs 2–4× faster than FP16. Apple ANE is natively INT8. Qualcomm HTP is INT4-native—the key to on-device LLMs.

Live: FP32 to INT8 Post-Training Quantization (PTQ)

Map continuous 32-bit float distributions into 256 discrete 8-bit bins using scale factor $S = \frac{\beta - \alpha}{255}$.

Memory Compression: 75% Saved

Quick answers

What is model quantisation?

Model quantisation reduces the numerical precision of neural network weights and activations from high-precision floats (FP32) to lower-precision integers (INT8 or INT4). This reduces memory footprint by 4–8×, accelerates inference on integer-optimised hardware, and enables running large models on resource-constrained devices.

What is the difference between PTQ and QAT?

Post-Training Quantisation (PTQ) quantises a pre-trained model without additional training, using a calibration dataset to set quantisation ranges—fast but lower accuracy. Quantisation-Aware Training (QAT) simulates quantisation during training so the model learns to compensate for precision loss, achieving higher accuracy at the cost of additional training compute.