Skip to the document
Madhuopen lab
Outcome School · Inference and serving21 min read

by Amit Shekhar · 17 June 2026

How does vLLM work?

How vLLM works. We will also see why we need it, how it manages memory so cleverly, and where it is used in the real world to serve large language models to many users at once.

4,007 words#llm#ai#system-design4 recall cards

How does vLLM work?
Read on Outcome School ↗then come back to lock it in
Before you read, guess

What problem does the KV cache introduce?

Ten seconds, a guess, then read — a wrong guess still makes the answer stick.

What this article covers

  1. What is serving an LLM
  2. A quick recap of prefill, decode, and the KV cache
  3. The problem: the KV cache eats GPU memory
  4. Why naive serving wastes memory
  5. What is vLLM
  6. PagedAttention, the core idea
  7. How PagedAttention shares memory
  8. Continuous batching
  9. The OpenAI-compatible API server
  10. The benefits of vLLM
  11. vLLM in the real world

The article lives on outcomeschool.com. Read it there, then come back: the tutor in the margin has read it and will answer questions, and the questions below check what stayed.

Before you go

In one sentence, what was this chapter about?

From memory, without scrolling up. Writing it is what makes it yours; the grade is only to show you what you had.

How sure?