Skip to the document
Madhuopen lab
Outcome School · Transformers and architecture19 min read

by Amit Shekhar · 10 June 2026

How does Token Streaming work?

How Token Streaming works. We will also see why we need it, how the server and the browser talk to each other to make it happen, and where it is used in real systems like ChatGPT and Claude.

3,675 words#llm#ai#system-design4 recall cards

How does Token Streaming work?
Read on Outcome School ↗then come back to lock it in
Before you read, guess

What must be understood to grasp the concept of streaming?

Ten seconds, a guess, then read — a wrong guess still makes the answer stick.

What this article covers

  1. What is token streaming
  2. A quick recap of how an LLM generates text
  3. Why we need streaming at all
  4. What is SSE
  5. How the HTTP connection stays open
  6. The format of a streamed message
  7. A full walkthrough from server to screen
  8. The [DONE] marker that ends the stream
  9. SSE vs WebSockets
  10. Token streaming in the real world

The article lives on outcomeschool.com. Read it there, then come back: the tutor in the margin has read it and will answer questions, and the questions below check what stayed.

Before you go

In one sentence, what was this chapter about?

From memory, without scrolling up. Writing it is what makes it yours; the grade is only to show you what you had.

How sure?