Karpathy's three layers — and how they show up in this lab

June 9, 2026

At Sequoia’s AI Ascent 2026 conference, Andrej Karpathy — former head of AI at Tesla, founder of Eureka Labs — laid out a framework for working effectively with AI agents. He broke it into three layers: the spec, the verifier, and the environment.

I watched a summary, read his own write-up, and realized this lab has been accidentally implementing all three for months. Here’s what that looks like in practice.

The problem Karpathy is solving

The foundation of his talk is what he calls the verifiability thesis: traditional software automates what you can specify in code. LLMs automate what you can verify. Because frontier labs train models with reinforcement learning and verification rewards, models get extremely good at tasks with measurable outputs — code that compiles, math that checks out — and stay rough on context-driven tasks that lack clear signals.

The car-wash example: ask any frontier model whether to drive or walk to a car wash 0.05 km away and it’ll tell you to walk. It has no signal that you need the car in the car wash. That’s not stupidity — it’s the gap between computational power and contextual understanding. Your job, as the human, is to close that gap.

His three layers are how you do it at scale.

Layer 1: The spec

The spec is how you transfer your understanding — your goals, constraints, and context — into something an agent can act on reliably. Karpathy’s point: plan mode and one-shot prompts aren’t enough. You have to work with the model to build a detailed spec before any real work starts.

In this lab, the spec is the CLAUDE.md file. It’s injected automatically into every Claude Code session before the first message. It contains:

• The full network topology (VLANs, IPs, which interface is mgmt vs. data plane)
• Every operational guardrail that matters — never commit secrets, never drop infra without an explicit instruction, always use internal Vault addresses for writes
• Canonical rules like “Terraform provider auth uses api_token, not username/password” — the kind of thing that costs you a failed apply to learn the hard way

Without it, Claude would make reasonable but wrong assumptions on every session. With it, the context that lives in my head lives in the spec, and the agent can act on it.

Karpathy’s advice: be agile with specs. Small scopes, clear checkpoints, review and adjust. Don’t hand an agent a waterfall task and expect a clean result.

Layer 2: The verifier

The verifier is everything you do to close the feedback loop. Boris Cherney, who built Claude Code, put it plainly: if Claude has a feedback loop, it will two to three times the quality of the final result.

In this lab, verification is built into the workflow in a few concrete ways:

Hermes reviews its own work before I see it. Hermes — a local agent running Nous Hermes on VM 302 — writes homelab documentation and opens pull requests on Gitea. Claude Code reviews and merges. Neither agent self-merges. That two-stage loop catches drift between what Hermes knows and what’s actually running.

Sessions end with a structured sign-off. The /done command runs a checklist: Obsidian synced, changes committed, Notion updated, repos clean. It forces a verification pass before context is lost.

External signal where it matters. After infrastructure changes, the next step is always checking something real — docker ps, ss -tlnp, a Prometheus target query — not trusting the agent’s own report of success.

This is Karpathy’s point about the robot librarian: it doesn’t know when it’s missing a book. The only lever you have is designing verification into the loop, not hoping the output is right.

Layer 3: The environment

The environment is the persistent, compounding layer — the workshop that improves every time you use it. Karpathy identifies three pieces: a CLAUDE.md, a local knowledge base, and custom skills.

The CLAUDE.md is covered above. The other two:

The knowledge base. Karpathy published a GitHub Gist in April 2026 describing an LLM wiki pattern: a raw/ directory of immutable source material, and a wiki/ directory of LLM-maintained markdown files that the model incrementally compiles, links, and lints. The model writes; you direct.

This lab runs exactly this pattern. Hermes has an external/ directory with three live sources: the Obsidian homelab wiki (synced from Gitea every 30 minutes), the IaC repository (Terraform and Ansible), and 800+ Notion pages synced nightly. When you ask Hermes about a host, it greps local files first — no live API calls, no hallucinated specs, just the actual documented state of the lab.

Custom skills. The /done command, /homelab-triage, an auth-check sequence — each is a reusable skill that runs the same way every time. Karpathy’s framing: if you do something repeatedly, encode it as a skill and run water through it until it stops leaking.

The one gap

Karpathy makes a distinction that’s easy to miss: there’s a difference between a rule in CLAUDE.md and a hard rule enforced at the tool layer. A CLAUDE.md rule is a request. The model can ignore it — and occasionally will.

Hard rules require hooks. A pre-tool-use hook that checks the file path before any Write or Edit runs literally prevents certain edits from happening, regardless of what the prompt says. This lab has a gitleaks pre-commit hook that blocks secrets from entering git, but no path-based write protection yet. That’s the next addition.

The part that sticks

Karpathy ended the talk with the question: what still matters when intelligence gets cheap?

You can outsource your thinking, but you can’t outsource your understanding.

The spec, the verifier, the environment — all three are centered on your understanding of the system. The agent handles recall and execution. You handle judgment, direction, and knowing when something looks wrong. That division of labor is what makes the whole thing work.

← all posts