Andrej Karpathy: Building Intuition Through Code
Karpathy's approach: build tiny versions of everything from scratch. Why his pedagogy produces the best AI intuition, and which of his resources are non-negotiable.
Who He Is
Andrej Karpathy co-founded OpenAI, led Tesla Autopilot, and in May 2026 joined Anthropic's pre-training team. He coined 'vibe coding' — a term Collins English Dictionary named Word of the Year 2025. But his most durable contribution is pedagogical: a body of work that teaches AI through code you build yourself.
Core Thesis
Build the simplest possible version of everything from scratch. If you can't implement it, you don't understand it.
Key Themes
- Learn by reimplementing — nanogpt, minbpe, micrograd all teach by stripping a concept to its minimum viable form
- Intuition before formalism — Karpathy teaches the shape of an idea before the math
- Vibe coding as acceleration — natural language → code as the new interface for software development
- Pre-training as the core — his research focus is always on what happens at the frontier of scale
- Teach in public — Neural Networks: Zero to Hero has 5M+ views because he explains what he genuinely finds confusing
Essential Reading / Watching
| Resource | Format | Why It Matters |
|---|---|---|
| Neural Networks: Zero to Hero | YouTube series | Build GPT-2 from scratch in Python. The best practical transformer education that exists. |
| nanogpt | GitHub repo | ~300 lines of GPT-2. Read it before reading any framework code. |
| minbpe | GitHub repo | BPE tokenizer from scratch. You'll never be confused about tokenization again. |
| The spelled-out intro to transformers | Blog + code | Attention mechanism explained with real matrix math before any abstraction. |
| Vibe Coding (karpathy.ai) | Blog post | The original coinage and thesis: natural language as the new programming interface. |
What to Question
Karpathy's pedagogy optimises for intuition over breadth. The 'build from scratch' method is powerful for fundamentals but doesn't address systems thinking — how components interact in production, how failures cascade, how to evaluate at scale. His work is the foundation, not the ceiling.
Try it interactively
GenAI Systems Lab is a free platform for AI engineers — configure real failure modes, break things, and build the judgment that gets you hired.
Open GenAI Systems Lab →