We Ran Static Quantisation and QAT Too. Both Are Still Slower Than float32.

The follow up everyone asked for, on one machine, with the numbers

By | 7 min read | Level: Intermediate to Advanced | Category: Deep Learning
float320.091
ms against
best int80.229

CPU · batch 1 · one thread · median of 300 passes · PyTorch 2.11.0+cpu · onednn

The original piece measured dynamic int8 quantisation and found it slower than float32. The reply, every time, was the same: dynamic is the weakest of the three options, use static quantisation or train with quantisation in the loop and it will be faster.

Fair objection. So we measured those too.

The objection, stated properly

There are three ways to get an int8 model in PyTorch and they differ in when the activation ranges are decided.

Dynamic quantises the weights ahead of time and works out the activation range on every single call. That per call work is real, and it is the obvious suspect for the original result.

Static observes the activation ranges once, on calibration data, and bakes them in. Nothing is computed per call. If the overhead was the range finding, static should win.

Quantisation aware training simulates the rounding during training, so the weights learn to survive it. It is the most expensive to produce and usually the most accurate.

What we measured

Table 1 · All four arms · CPU, batch 1, one thread, median of 300 passes
accuracy size KB median ms p90 ms against float32
float32 0.9791 3223.5 0.0914 0.1103 leader
dynamic int8 0.9788 812.7 0.2294 0.2687 2.5x slower
QAT int8 0.9793 830.7 0.2367 0.2790 2.6x slower
static int8 0.9786 830.7 0.2397 0.4150 2.6x slower
Every int8 path lands within 0.011 ms of the others. The choice between them did not matter; the choice to quantise at all did.

Static quantisation was not faster than dynamic. On this machine it was marginally slower, and the gap between the three int8 arms (0.0103 ms) is smaller than the run to run variation on any one of them.

All three int8 paths are roughly two and a half times slower than float32 on this CPU. The objection was reasonable and it turned out not to be the explanation.

What QAT actually bought

Table 2 · Accuracy · 10,000 held out digits, five epochs each
accuracy against float32
QAT int8 0.9793 +0.0002
float32 0.9791 leader
dynamic int8 0.9788 -0.0003
static int8 0.9786 -0.0005
A spread of 0.0007 across four arms, on 10,000 samples. That is seven digits out of ten thousand and it is noise, not a ranking.

Almost nothing, on this problem. QAT came top by 0.0002, which on 10,000 samples is two digits.

That is worth stating plainly because the first version of this experiment got it wrong in a way worth admitting. QAT was trained for five epochs and float32 for three, and QAT duly looked more accurate. It was not more accurate; it had two more epochs. Every arm in the table above trains for five. An unfair control is worse than no control, and it is an easy one to ship without noticing.

Why none of it was faster

The honest answer is that the win from int8 arithmetic has to be larger than the cost of moving between representations, and at this size it is not.

Every quantised layer converts its input to int8, does the matrix multiply, then converts back out. At batch 1 the multiply is small, so the conversion is a meaningful share of the work rather than a rounding error. int8 arithmetic is faster per operation and there are not enough operations here for that to pay.

That also explains why static did not help. Static removes the range calculation, and the range calculation was never the expensive part. The conversions are, and static does those too.

The file did shrink: 74 per cent smaller, from 3224 KB to 831 KB. If the constraint is memory or download size, quantisation delivers. If the constraint is latency at batch 1, on this hardware, it does not.

Where this result does not apply

Four limits, and they are load bearing.

Batch size 1. This is the latency sensitive setting. At larger batches the multiply grows and the conversion overhead is amortised. int8 generally does win there.

This model is small. 824,550 parameters, three linear layers. On a model where the weights do not fit comfortably in cache the picture changes, because int8 weights mean a quarter of the memory traffic.

One backend. This build of PyTorch exposes only onednn. Results with fbgemm or qnnpack may differ, and anyone seeing a different number on a different backend is not contradicting this one.

Linear layers only. Quantised convolution takes different kernels. Nothing here says anything about a convolutional network.

The point is not that quantisation is useless. It is that "quantise it to make it faster" is a hypothesis about your model on your hardware at your batch size, and it takes about an hour to test.

Run it yourself

Same machine, same process, same seed, five epochs per arm, 300 timed passes after 50 warmup passes, threads pinned to one so the measurement is of the kernel rather than of the scheduler. Two independent runs agreed on the ordering and on the size of the gap.

If you are measuring this kind of thing on a system that has to go to production, the same discipline applies to agents and language models: fix the cases, fix the thresholds, and diff the runs. We wrote an open harness for that at r4agent.