Skip to the document
Madhuopen lab
Outcome School · Transformers and architecture10 min read

by Amit Shekhar · 31 August 2026

Decoding Medusa

Medusa, a simple way to make a language model generate text 2 to 3 times faster by giving it several extra heads that guess multiple future tokens at once.

1,851 words#llm#ai#machine-learning4 recall cards

Decoding Medusa
Read on Outcome School ↗then come back to lock it in
Before you read, guess

Why is standard generation slow regarding GPU utilization?

Ten seconds, a guess, then read — a wrong guess still makes the answer stick.

What this article covers

  1. What is Medusa
  2. Why text generation is slow
  3. A quick recap of speculative decoding
  4. The problem with needing a draft model
  5. The big idea: many heads on one model
  6. How tree attention checks many guesses at once
  7. The math behind the speedup with small numbers
  8. The results
  9. How Medusa lives on today

The article lives on outcomeschool.com. Read it there, then come back: the tutor in the margin has read it and will answer questions, and the questions below check what stayed.

Before you go

In one sentence, what was this chapter about?

From memory, without scrolling up. Writing it is what makes it yours; the grade is only to show you what you had.

How sure?