Skip to the document
Madhuopen lab
Outcome School · ML foundations and math16 min read

by Amit Shekhar · 20 April 2026

Math Behind Cross-Entropy Loss

The math behind Cross-Entropy Loss with a step-by-step numeric example.

3,027 words#math#llm#ai#machine-learning4 recall cards

Math Behind Cross-Entropy Loss
Read on Outcome School ↗then come back to lock it in
Before you read, guess

What is the simplified cross-entropy loss formula for one-hot labels?

Ten seconds, a guess, then read — a wrong guess still makes the answer stick.

What this article covers

  1. The Big Picture
  2. What is Cross-Entropy
  3. The Cross-Entropy Loss Formula
  4. Why We Take the Negative Log
  5. Binary Cross-Entropy Loss
  6. Categorical Cross-Entropy Loss
  7. Step-by-Step Numeric Example
  8. Cross-Entropy Loss for Language Models
  9. The Gradient of Cross-Entropy Loss

The article lives on outcomeschool.com. Read it there, then come back: the tutor in the margin has read it and will answer questions, and the questions below check what stayed.

Before you go

In one sentence, what was this chapter about?

From memory, without scrolling up. Writing it is what makes it yours; the grade is only to show you what you had.

How sure?