Integer quantisation replaces a real number with a small integer using , where the step size and the zero-point are computed from the tensor’s own minimum and maximum.
Everything else follows from those two constants by arithmetic: the error is bounded by , each extra bit buys about 6.02 dB of signal-to-noise, and “4-bit” weighs about 0.578 bytes per weight rather than 0.5.
This page does all of that arithmetic in front of you.
The affine map, and where s and z come from A float carries its own exponent, so it can represent both 1e-8 and 1e8.
An integer cannot.
Every floating-point format spends bits on that exponent; integer quantisation spends none, and buys the range back by attaching one shared scale to a whole block of numbers.
That is the entire idea.
The rest is bookkeeping.
The two formulas for and are not conventions, they are the unique solution to two requirements: that maps to and maps to .
Solving those two equations simultaneously gives exactly the lines above.
Worked on a real-shaped activation tensor.
Take the output of a GELU somewhere in the middle of a network, with an observed minimum of and a maximum of , quantised to uint8: Zero surviving exactly is the reason is an integer rather than a real offset.
Padding tokens, masked positions and every output of a ReLU are exactly zero, and a scheme that turned them into 0.003 would leak a small signal into every place the network relies on nothing being there.
The price of that integer is small and worth naming, because almost nobody does.
Rounding from 12.1090 to 12 shifts the whole representable window: Symmetric or asymmetric, and why it differs by tensor Symmetric quantisation fixes and derives the scale from the absolute maximum alone, so the map collapses to and a dequantisation of .
Asymmetric keeps the zero-point.
The choice is not a matter of taste; it falls out of what the two tensors look like and out of what the matmul has to compute.
Weights take the symmetric form because of what happens when the matmul is expanded.
Substitute both dequantisation formulas into a single dot product over terms and multiply out: Term 2 is the expensive one.
It depends on the activations, so it cannot be precomputed, and it costs a full extra reduction over per output.
Setting deletes it.
That is the whole argument, and it is an argument about the shape of the matmul, not about the distribution of weights — although the distribution cooperates, since a trained weight row is roughly zero-centred and loses little to a symmetric range.
Activations go the other way because they are one-sided.
A ReLU or a GELU output is mostly non-negative, and forcing symmetry throws away the levels below zero: A second, quieter reason activations differ: their range is not known at load time.
It is either measured on a calibration set beforehand (static quantisation, where an unrepresentative calibration set is the usual cause of a mysterious accuracy drop) or computed per batch at run time (dynamic quantisation, which is more robust and costs a min/max reduction over every tensor).
Weights have no such problem: their range is a fact about a file.
Step size, the half-step bound, and 6.02 dB per bit The step size is the whole error story.
Rounding to the nearest multiple of cannot be wrong by more than half a step, so for any inside the representable window: That bound holds only inside the window.
Outside it there is no bound at all: a value of 0.9 in a tensor scaled for 0.212 clamps to 0.212 and the error is 0.688, which is 824 half-steps.
Clipping error and rounding error are different failures with different fixes, and conflating them is the most common mistake in tuning a quantisation scheme.
Now the signal-to-noise result, derived rather than quoted.
Model the rounding error as uniformly distributed across one step, which is accurate whenever the signal moves over many steps: So the constant is , and the mechanism behind it is one line: an extra bit halves , halving quarters , and a fa