Skip to the document
Madhuopen lab
Outcome School · Other AI topics11 min read

by Amit Shekhar · 25 August 2026

AI Is Only as Good as Our Definition of Done

One simple idea, AI is only as good as our definition of done, which means the clear line we draw between a task that is finished and a task that is not. We will also see what a definition of done really is, why AI feels magical on the tasks that a machine can check, why it feels unreliable on the tasks that only a human can judge, how the loop that AI runs behind the scenes creates this exact gap, and how we can write a definition of done that turns a vague task into a checkable one.

2,026 words#llm#ai#system-design4 recall cards

AI Is Only as Good as Our Definition of Done
Read on Outcome School ↗then come back to lock it in
Before you read, guess

What requirement ensures a function behaves predictably for specific inputs?

Ten seconds, a guess, then read — a wrong guess still makes the answer stick.

What this article covers

  1. What is a definition of done
  2. Tasks where the definition of done is exact
  3. Tasks where the definition of done is fuzzy
  4. Why AI is so strong exactly where we can measure
  5. How to write a better definition of done

The article lives on outcomeschool.com. Read it there, then come back: the tutor in the margin has read it and will answer questions, and the questions below check what stayed.

Before you go

In one sentence, what was this chapter about?

From memory, without scrolling up. Writing it is what makes it yours; the grade is only to show you what you had.

How sure?