Skip to the document
Madhuopen lab
Outcome School · Retrieval and RAG11 min read

by Amit Shekhar · 27 March 2026

KV Cache in LLMs

KV Cache - where K stands for Key and V stands for Value - and why it is used in Large Language Models (LLMs) to speed up text generation.

2,032 words#llm#ai#machine-learning4 recall cards

KV Cache in LLMs
Read on Outcome School ↗then come back to lock it in
Before you read, guess

Why is processing previous tokens inefficient in standard LLM inference?

Ten seconds, a guess, then read — a wrong guess still makes the answer stick.

What this article covers

  1. How LLMs Generate Text
  2. What Happens Inside the Model
  3. The Problem: Repeated Computation
  4. The Solution: KV Cache
  5. Why Only Key and Value Are Cached, Not Query
  6. How Much Faster Does It Get
  7. The Trade-Off: Speed vs Memory

The article lives on outcomeschool.com. Read it there, then come back: the tutor in the margin has read it and will answer questions, and the questions below check what stayed.

Before you go

In one sentence, what was this chapter about?

From memory, without scrolling up. Writing it is what makes it yours; the grade is only to show you what you had.

How sure?