Skip to the document
Madhuopen lab
Outcome School · Inference and serving20 min read

by Amit Shekhar · 13 September 2026

Prefill-Decode Disaggregation

Prefill-Decode Disaggregation, a way of running a large language model where the reading of the prompt and the writing of the answer happen on separate machines. We will also see how an LLM answers a request in two phases, what the KV Cache is, why the two phases need different things from the hardware, what goes wrong when both run on the same machine, how disaggregation solves it step by step, its advantages and disadvantages, and where it works well and where it is overkill.

3,991 words#llm#ai#system-design4 recall cards

Prefill-Decode Disaggregation
Read on Outcome School ↗then come back to lock it in
Before you read, guess

Which phase generates output sequentially and consumes significant memory?

Ten seconds, a guess, then read — a wrong guess still makes the answer stick.

What this article covers

  1. How an LLM answers a request
  2. What is the KV Cache?
  3. Prefill is compute-heavy, Decode is memory-heavy
  4. The problem when both run on the same GPU
  5. TTFT vs TPOT
  6. The naive approaches and their issues
  7. What is Prefill-Decode Disaggregation?
  8. How Prefill-Decode Disaggregation works
  9. Walkthrough of one request
  10. Advantages of Prefill-Decode Disaggregation
  11. Disadvantages of Prefill-Decode Disaggregation
  12. Where it works well and where it is overkill
  13. Co-located vs Disaggregated serving
  14. Summary

The article lives on outcomeschool.com. Read it there, then come back: the tutor in the margin has read it and will answer questions, and the questions below check what stayed.

Before you go

In one sentence, what was this chapter about?

From memory, without scrolling up. Writing it is what makes it yours; the grade is only to show you what you had.

How sure?