The 2025 State of AI (through Karpathy’s lens) -
it’s not “the year of agents.” It’s the decade.
Recently, Andrej Karpathy was interviewed on the Dwarkesh podcast (I'll put a link in the comments), and this is what he said on the current state of AI agents:
1) Creating AI agents that work reliably is a project that will take up to 2035 to complete.
We don’t yet have the scaffolding for coworker-level agents: durable memory, reliable multimodal perception, continual learning, robust tools/OS control.
This is a marathon of engineering, not a quarter’s roadmap.
2) Software progression from 1.0 to 2.0 to 3.0
1.0: humans write code
2.0: data writes weights
3.0: we program via language + LLMs
Don’t skip the representation era.
Strong “cognitive core” comes first, agents second.
3) We’re building “ghosts,” not animals.
Models simulate human behavior from data. No instincts. No body.
Expect brilliance in patterning, less so in grounded, embodied common sense (at least for now).
4) Reasoning lives in the context window.
Weights are kind of like long-term memory.
Context is kind of like working memory where thinking happens.
Feed richer context, get sharper reasoning.
Retrieval is way better than wishful prompting.
5) Only part of the ‘brain’ exists
We have Cortex-like patterning.
We don't have Hippocampus-like memory consolidation, emotion/instinct, coordination.
That’s why current systems “think,” but don’t truly embed.
6) Building beats hand-wavy prompting
Use AI to accelerate, not abdicate. Autocomplete helps but novel system design still needs humans who understand what they’re building.
7) RL is inefficient, process beats outcome-only rewards.
Next step: process-based supervision and reflection loops.
Teach models to review, self-correct, and generate useful training data.
8) Forgetting can improve generalization.
Humans generalize because we forget. Models that memorize everything risk rigidity. Curate, prune, and reinforce abstraction, not trivia.
9) Bigger isn’t always better.
Clean, high-quality data + smart architecture can outperform “just scale it.” A lean, well-trained cognitive core can feel startlingly capable.
So, to summarize, here are a few practical takeaways:
* Design for context: Retrieval + tool use + structured memory.
* Instrument the process: Critique, reflect, and fine-tune steps, not just outputs.
* Curate ruthlessly: Data hygiene is a moat.
* Ship scaffolding: Memory, tool orchestration, reliability.
Think in decades, execute weekly, and compound small real improvements.
Let's not forget that we’re mid-build, not at the finish line.
The transformer will likely be with us a while, just refined.
The winners will be the ones who build patiently, instrument relentlessly, and curate obsessively.
Let me know your thoughts in the comment section
![]()
1
1 Comment
