The Feynman Technique for Debugging Your Own Understanding

Explain It Out Loud

Richard Feynman, a Nobel Prize-winning physicist, had a simple method for testing whether he understood something: he'd explain it as if teaching a bright twelve-year-old. No jargon. No hand-waving. Just clear reasoning.

If he couldn't explain it simply, he didn't understand it.

This is the Feynman Technique, and it's brutally effective for learning AI and ML concepts. The reason: when you write or speak explanations, your gaps in understanding become obvious. You can read a paper on transformers and feel like you understand. But try to explain attention to a friend without notes, and suddenly you're stuck. That's not a failure; that's useful information. Your stuck point is exactly where you need to study.

The technique works in a loop:

1. Explain simply (write or talk out loud)
2. Find where you got stuck (the gap is your study list)
3. Relearn that gap (go back to sources)
4. Re-explain from scratch (now smoother)

Repeat until no gaps remain.

Step 1: Explain It Out Loud (Or Write It Out)

Pick a concept you've recently learned. Let's say you've been studying gradient descent.

Set a timer for 5–10 minutes. Open a blank document or sit with a rubber duck (literally—some people talk to a duck). Explain the concept as if your audience knows about math and code, but not AI.

Here's an attempt:

"Okay, so gradient descent. When you train a neural network, you start with random weights. You want the network to predict things correctly, so you define a loss function—basically, 'how wrong are you?' Lower loss is better. Gradient descent is how you make the loss lower. You compute the gradient of the loss—which direction makes it worse and by how much—and you step in the opposite direction. Like going downhill in the fog: you can't see the bottom, but you can feel which way is downhill, so you take a step down. Repeat. Eventually you're at the bottom (or stuck in a valley). That's the minimum. Your weights are now trained."

This explanation is okay. You got the basic idea across. But read it again. Where did you hesitate or hand-wave?

• "Like going downhill in the fog" is an analogy, not an explanation. It feels intuitive, but what does "feeling the gradient" actually mean mathematically?
• "Eventually you're at the bottom (or stuck in a valley)" glosses over local minima. You didn't explain when that's a problem or why deep networks seem to avoid it.
• You didn't mention learning rate. Why does step size matter? What happens if the step is too big?

These vague spots are your gaps.

Step 2: Identify the Gaps

Go back to your explanation and mark the places where you:

• Used metaphors instead of concrete descriptions
• Said "and so on" or "et cetera"
• Used jargon you didn't define (or explained with more jargon)
• Hesitated or paused while explaining
• Couldn't answer a follow-up question

In your gradient descent explanation, the gaps are:

1. What is a gradient, mathematically? You said "direction that makes it worse," but direction in which space? With how many dimensions?
2. What determines the step size, and why does it matter? You mentioned "downhill" but not convergence speed or stability.
3. Why don't we always get stuck in local minima? Your explanation assumed we'd find a good solution, but didn't explain why.

These become your study list. You've now converted a vague sense of "I understand gradient descent" into specific, actionable questions.

Step 3: Relearn the Gap (Targeted Study)

This is where spaced repetition and textbooks come in, but with focus.

Instead of re-reading a 10-page chapter on optimization, you target:

• One textbook section on computing gradients (backpropagation)
• One paper or blog post on the implicit bias of SGD (why local minima aren't as bad as we thought)
• One tutorial on learning rate schedules

You're reading with a specific question in mind, not trying to re-learn everything.

For gradient descent specifically:

Gap 1: Read the backpropagation chapter. You'll see ∂L/∂w for each weight. You'll understand that the gradient is a vector; each component tells you how much to change that weight.

Gap 2: Read about learning rates. You'll see that if α (learning rate) is 0.1 and ∂L/∂w = [0.5, −0.2, 0.3], then w := w − 0.1 × [0.5, −0.2, 0.3]. Step size directly scales the gradient. Too big → overshoots. Too small → slow progress.

Gap 3: Read a paper like "On the Generalization of Equivariance and Convolution" or blog posts on loss landscape geometry. You'll learn that:

• Neural networks have many local minima.
• But most local minima have similar loss (bad minima are rare).
• SGD with noise (randomness in batching) avoids bad minima.
• So local minima aren't the real problem; poor generalization is.

Step 4: Re-Explain from Scratch

Now explain gradient descent again, without looking at your previous attempt.

"Gradient descent trains a neural network by minimizing the loss function. The loss measures how wrong the predictions are. To minimize loss, we compute the gradient—a vector where each element is ∂L/∂w_i, the derivative of loss with respect to each weight. The gradient points in the direction of steepest increase. So we move opposite to the gradient: w := w − α∇L, where α is the learning rate (step size). If α is too big, we overshoot and the loss oscillates. If α is too small, training is slow. We repeat this update until the gradient is near zero, which means we've found a local minimum. In practice, local minima aren't a big problem for deep networks because most minima have similar loss. The real challenge is generalization—the model might fit the training data well but perform poorly on test data. That's where regularization comes in."

This explanation is tighter. You've integrated what you learned about learning rates and local minima. You've moved the conversation toward the next gap (generalization and regularization). You're spiraling deeper, not just repeating.

Repeat this loop for each concept. After 3–4 cycles of explain → gap → study → re-explain, the concept moves from "something I half-understand" to "something I can teach."

Applied Example: Understanding Attention—Feynman Worksheet

Let's work through attention from scratch using a structured worksheet.

First Explanation & Gap Identification (5 minutes)

Your explanation (pretend you're explaining to a friend):

"Attention is a mechanism that lets a model focus on the most relevant parts of the input. In a transformer, each word attends to other words in the sequence. You compute attention weights—how much each word should pay attention to each other word—and use those weights to combine word embeddings. It's like, you're reading a sentence, and you focus more on some words and less on others."

Gaps you notice (read it back, mark the vague spots):

1. "How do you compute attention weights?" ← Hand-waved with "focus"; no concrete process
2. "How much each word should pay attention?" ← Vague; no math
3. "Combine word embeddings" ← How exactly? Weighted average? Concatenation?
4. "Why should transformers use this instead of just processing all words equally?" ← Not answered at all

Scored clarity: 3/10 (intuitive, but almost no concrete detail)

Targeted Study (20 minutes)

Resources to read:

• "Attention Is All You Need" paper, Section 3.1 (Scaled Dot-Product Attention), 2 pages
• Blog post or tutorial explaining Attention(Q, K, V) = softmax(QK^T / √d_k)V, 1 page

After study, write down:

• What is Q? Learned linear projection of the input embedding
• What is K? Same; learned linear projection
• What is V? Same; learned linear projection
• The formula: softmax takes raw attention scores (QK^T / √d_k) and normalizes them to a probability distribution. Then weighted average of V using those probabilities.
• Why √d_k? Prevents large dot products (which are more likely when d_k is large) from pushing softmax into the flat tail where gradients vanish

Second Explanation (5 minutes)—Now Much Sharper

"Attention computes a weighted combination of values (V) using weights derived from similarity between queries (Q) and keys (K). Specifically: (1) Compute raw scores QK^T (dot product between each query and each key). (2) Scale by 1/√d_k to prevent numerical instability. (3) Apply softmax to turn scores into a probability distribution (weights). (4) Take the weighted sum of V using these weights. This lets the model attend to different positions based on learned similarity, not fixed rules."

Scored clarity: 7/10 (concrete formula, but gaps remain: Why Q, K, V split? Why this form specifically? Where do these projections come from?)

Third Explanation (after more study, maybe another 20 minutes)

You read more on why multi-head attention exists, why positional encodings are needed, etc. Then re-explain:

"Attention computes relevance-weighted context. Given input embeddings, we project them into Q (query), K (key), and V (value) spaces using learned linear layers. For each query position, we compute a weighted average of values, where weights are determined by the similarity (dot product) between the query and all keys. Softmax normalizes these raw scores into a probability distribution. The √d_k scaling prevents numerical issues. Crucially, this is position-agnostic: attention depends only on similarity, not distance. We use multiple attention heads in parallel so the model can attend to different types of information (e.g., one head attends locally, another attends globally). Finally, positional encodings are added to embeddings to inject position information, since plain attention ignores word order."

Scored clarity: 9/10 (concrete, mathematical, explains design choices)

Feynman Worksheet Template (Use This)

# Feynman Technique Worksheet: [Concept Name]

## Iteration 1: First Explanation
**Time:** [5 min]
**Your explanation (write or record audio):**
[Your attempt]

**Clarity score:** __/10

**Gaps (vague words, hand-waves, unanswered questions):**
1. [Gap 1]
2. [Gap 2]
3. [Gap 3]

---

## Iteration 2: Study & Re-explain
**Study materials reviewed:**
- [Source 1, time spent]
- [Source 2, time spent]

**Key learnings from study:**
- [Concrete detail 1]
- [Concrete detail 2]

**Second explanation:**
[Updated attempt]

**Clarity score:** __/10

**Remaining gaps:**
1. [If applicable]

---

## Iteration 3: Deep Study & Final Explanation
**Additional study:**
- [Deeper source]

**Final explanation:**
[Final version, should be much sharper]

**Clarity score:** __/10

**Confidence:** Yes / Partial / No

Why This Works

The Feynman Technique works because explaining forces generation, not passive reading. Your brain has to organize and output what it knows. Gaps become obvious.

| Learning Method | Brain Activity | Gaps Found | Retention | |-----------------|----------------|------------|-----------| | Reading a paper | Passive input | Few; subtle gaps hidden | 40% after 1 week | | Watching a lecture | Passive input + some processing | Few | 25% after 1 week | | Feynman technique | Active generation + gap finding + targeted study | Many; explicit and actionable | 80% after 1 week |

The superiority of Feynman comes from: (1) generation forces organization, (2) gaps are explicit not implicit, (3) gaps drive targeted, efficient study.

Example: Using Feynman to Debug a Misbehaving Model

Here's a real scenario where Feynman technique would help:

# Problem: Model Accuracy Plateaus at 85% on Validation, Won't Improve

## First Explanation (your attempt to diagnose, 5 min)
"The model isn't learning well. Maybe the learning rate is too low? Or the model is underfitting. I should try adding more layers or using a better optimizer. Or maybe the data is bad."

Vague. Gaps: What does "not learning well" mean exactly? Is it underfitting or overfitting? How would you know?

## Identify Gaps
1. How do you distinguish underfitting from overfitting?
2. What specifically indicates a learning rate problem?
3. What would change if you added layers vs. changed the optimizer?

## Targeted Study (30 min)
Read: "Is your model overfitting or underfitting?" blog post + Section on learning curves
Learn: Plot train vs. validation loss. If train loss is high and validation loss is high, it's underfitting. If train loss is low and validation loss is high, it's overfitting.

## Re-explain
"To diagnose why accuracy plateaus, I'd check: (1) Plot train vs validation loss curve. (2) If both are high and flat, it's underfitting—model can't fit training data. Add layers, increase capacity. (3) If train loss is low but validation loss is high/oscillating, it's overfitting—model fits training data too well. Add regularization, use dropout, or get more data. (4) If both losses are decreasing steadily but slow, learning rate might be too low. (5) If loss oscillates wildly, learning rate is too high."

Much more concrete. You can now actually debug.

This example shows how Feynman turns "the model is broken" into "the model is underfitting; I'll add capacity and regularization."

Clarity Progression Table: Batch Normalization

Here's how clarity improves over Feynman iterations:

| Iteration | Your Explanation | Clarity | Gaps Remaining | |-----------|-----------------|---------|-----------------| | 1st | "Batch norm normalizes input by subtracting mean and dividing by std. Makes training faster." | 4/10 | Which mean/std? Across what? Why faster? Different at train vs test? | | 2nd | "Batch norm computes mean/variance across batch. At train: batch stats. At test: running stats (EMA). Normalizes activations so next layer gets stable input." | 7/10 | Why different train vs test? How compute EMA? When NOT to use? | | 3rd | "Batch norm: Normalize (x - μ_batch) / √(σ_batch² + ε), then scale y = γx̂ + β (learnable). Fixes internal covariate shift. Train: batch stats, Test: running stats. Skip when: batch size < 8, sequential data, certain RNNs." | 9/10 | Advanced: initialization of γ/β, relationship to layer norm |

Why This Works

The Feynman Technique works because:

1. Speaking exposes gaps. Reading activates memory retrieval; speaking requires you to generate explanation, which is harder and more revealing.

2. Gaps are actionable. "I don't understand attention" is a vague problem. "I can explain attention intuitively but can't write out the math" is specific. It tells you exactly what to study.

3. The loop is cumulative. Each pass teaches you something, but more importantly, it teaches you what to study next. You're not guessing at what matters; your gaps are telling you.

4. Explanation forces simplicity. Jargon feels like understanding until you try to use it. Forcing yourself to explain in plain language (no "basically," no technical shortcuts) is the test.

Making It a Habit

Use the Feynman Technique once a week on concepts you've been studying:

1. Pick a concept you learned 3–5 days ago. Not so fresh that memorization still works. Not so old that you've forgotten it.

2. Explain it out loud (5 minutes). Record yourself on your phone if it helps you catch your own gaps.

3. Write down the gaps (2 minutes). Be specific: "I couldn't explain why learning rate matters" not "I didn't fully understand training."

4. Spend 15–30 minutes on targeted study of one gap. Don't re-read everything; target the gap.

5. Re-explain once more (3 minutes). You'll feel the difference immediately.

This takes 30–40 minutes per concept, and you can do 1–2 per week without much effort. Over a month, you'll have deeply understood 4–8 concepts instead of vaguely grasping 20.

Beyond Explanations: Testing Your Understanding

The Feynman Technique's strength is in identifying gaps through explanation. But explanations alone can be deceiving. You might explain something fluently and still not truly understand it.

Pair explanation with application:

After you've re-explained a concept clearly, do one of these:

• Implement it: Write code that uses the concept. If you understand gradient descent, implement SGD from scratch. Can you do it without looking at the original code?
• Teach it: Record yourself explaining to an imaginary student (or your friend). Teaching is harder than explaining in writing; it forces you to organize your thoughts linearly.
• Apply it to a different domain: You've learned gradient descent for neural networks. Can you explain how it applies to linear regression or SVM? If you understand the principle, you can transfer it.
• Predict what happens: You understand attention; now predict: "If I increase the number of attention heads, what happens to memory usage and training time?" Can you reason through it?
• Debug something: Read someone else's broken implementation and fix it using your understanding. This is the hardest test; it reveals gaps you didn't know you had.

If you can do any of these, you've moved from "understand the concept" to "can use the concept." That's the distinction that matters.

Scaling the Feynman Technique Across Your Learning

Over a month, apply the technique to 4–8 concepts (one per week). Over a year, that's 50–100 deeply understood concepts. This is how you build substantial knowledge.

The key is consistency, not intensity. One hour per week on the Feynman Technique, done every week for a year, compounds into real expertise. Binge-studying concepts for a weekend and then forgetting them doesn't.

Integration with your other learning systems:

• With spaced repetition: Turn your Feynman gaps into Anki cards. The gaps are exactly what you need to remember.
• With your learning log: Write a log entry for each concept you've Feynman'd. The re-explanation becomes your one-sentence takeaway.
• With projects: As you build, apply Feynman to concepts that come up. "I'm debugging why my model overfits; let me Feynman 'regularization' and understand it better."

The Feynman Technique isn't a standalone method; it's a complement to active learning (projects), repetition (spaced systems), and documentation (logging).

Common Mistake

The biggest mistake is using the Feynman Technique as a final exam rather than a learning tool. People try to explain a concept they've only skimmed once, feel bad when they can't, and give up. That's not the point. The point is to surface gaps so you can study them.

Another mistake: being too ambitious. Don't try to explain an entire architecture (like BERT) in 5 minutes. Explain one piece: "How does BERT's WordPiece tokenization work?" or "Why does BERT use masked language modeling instead of next-token prediction?" Narrow concepts are easier to explain clearly and more useful to understand deeply.

A third mistake: explaining to no one. Explaining to yourself in your head is easier than explaining out loud or writing it out, because your brain fills in gaps you didn't notice. Force yourself to say it aloud or write it down. The friction is the feature.