Andrej Karpathy is experiencing what he calls "post-AGI" firsthand. The renowned AI researcher revealed that autonomous agents are now iterating on his nanochat project, making continuous improvements without his intervention.

What Happened

Over a 12-hour period, AI agents generated 110 changes to the codebase, reducing validation loss from 0.862415 to 0.858039 for a d12 model—at no cost to wall clock time. The agents work on feature branches, test ideas, and merge them when successful, all autonomously.

The Technical Context

The update comes alongside meaningful progress on nanochat itself. Training a GPT-2 capability model now takes just 2 hours on a single 8XH100 node, down from roughly 3 hours a month ago. The biggest driver? Switching from FineWeb-edu to NVIDIA's ClimbMix dataset. Karpathy notes he tried Olmo, FineWeb, and DCLM—all led to regressions. ClimbMix worked out of the box, though he admits being "slightly suspicious about Goodharting."

The New Meta

Karpathy's real insight is meta: "the real benchmark of interest is: what is the research org agent code that produces improvements on nanochat the fastest?" Over the last two weeks, he's iterated more on optimizing agent flows than on the nanochat repo itself.

The Feeling

"ah yes, this is what post-agi feels like :) i didn't touch anything. brb sauna," he tweeted.

The takeaway: the line between human researcher and autonomous agent is blurring—and Karpathy is enjoying the view from the sauna.