by Amit Shekhar · 6 May 2026
Speculative Decoding
Speculative Decoding - what it is, why LLM generation is slow without it, how a small draft model and a big target model work together to produce tokens faster, the rejection sampling math that guarantees no quality loss, real numbers showing the 2x to 3x speedup, where it is used in production, and the trade-offs to watch out for.
Read on Outcome School ↗then come back to lock it in
Before you read, guessHow does speculative decoding accelerate generation using draft and target models?
Ten seconds, a guess, then read — a wrong guess still makes the answer stick.
What this article covers
- What problem does Speculative Decoding solve?
- The Big Picture
- Why is LLM generation slow?
- The core idea behind Speculative Decoding
- Step-by-step walkthrough
- The verification step
- Real numbers and speedup
- Where it is used
- Trade-offs
The article lives on outcomeschool.com. Read it there, then come back: the tutor in the margin has read it and will answer questions, and the questions below check what stayed.