by Amit Shekhar · 15 July 2026
How does Model Quantization work?
How Model Quantization works. We will also see why we need it, how it shrinks big AI models, and how it lets us run them on a laptop, a phone, or a single GPU.
Read on Outcome School ↗then come back to lock it in
Before you read, guessHow does quantization reduce the precision of stored numbers?
Ten seconds, a guess, then read — a wrong guess still makes the answer stick.
What this article covers
- What is Model Quantization?
- How numbers use bits (FP32, INT8, INT4)
- Why fewer bits means less memory and faster speed
- The core mechanic: scale and zero-point
- Symmetric vs Asymmetric Quantization
- Per-tensor vs Per-channel Quantization
- Post-Training Quantization (PTQ) vs Quantization-Aware Training (QAT)
- Weight-only vs Weight-and-activation Quantization
- The outlier problem in LLMs
- Popular methods: GPTQ, AWQ, bitsandbytes, GGUF / llama.cpp
- The accuracy trade-off and running LLMs locally
The article lives on outcomeschool.com. Read it there, then come back: the tutor in the margin has read it and will answer questions, and the questions below check what stayed.