What it does
autoresearch is a small open-source repository, published under the karpathy GitHub account, that turns a coding agent into an unattended machine learning researcher. The agent is given a compact single-GPU language model training setup. It changes the training code, trains for a fixed five minutes, checks whether the validation score improved, keeps or discards the change, and loops. People do not edit the Python files. Instead they write program.md, a Markdown instruction file, and point an agent such as Claude or Codex at the repository. Only train.py is edited by the agent, while prepare.py holds fixed data preparation and evaluation code. The metric is validation bits per byte, where lower is better. The fixed time budget works out to roughly twelve experiments an hour, or about a hundred overnight. It needs a single NVIDIA GPU (tested on an H100), Python 3.10 or newer and the uv package manager. The author says other hardware is not supported directly and links community forks for macOS, Windows and AMD. The project uses the MIT licence.
Capabilities
- Agent-driven loop: edit, train for five minutes, evaluate, keep or discard
- One editable training file keeps each change easy to review
- Fixed time budget makes experiments comparable on the same hardware
- Behavior steered by a Markdown instruction file, program.md
- Community forks for macOS, Windows and AMD GPUs
When to use it
- Trying autonomous, agent-driven experimentation on a small language model overnight
- Studying how to write instructions for an agent that runs a research loop
- Using a compact GPT training codebase as a base for your own agent experiments
