Towards self-driving codebases

Summary

The post describes research into scaling long-running autonomous coding agents, using a web browser as a complex benchmark and running thousands of agents continuously for one week. An initial single-agent plan failed because the model lost track, stopped prematurely, and stalled on complex details; shared-state coordination then introduced locking contention, while more structured planner, executor, worker, and judge roles improved ownership and throughput. The team built a Rust-based harness on one large Linux VM, logged agent messages and tool activity for replay and analysis, and iterated through designs that eventually favored a more dynamic system with freshness mechanisms and fewer overloaded roles. The research concludes that architecture, prompts, observability, explicit throughput trade-offs, and anti-fragile recovery matter, reporting linear scaling of token throughput with minimal harness overhead while humans retained taste, judgment, and direction.

Takeaways

1. 1

   A shared coordination file caused lock misuse and contention: 20 agents slowed to the throughput of 1–3, with most time spent waiting for locks.

2. 2

   Role separation gave a root planner or executor ownership of the broader goal while workers handled narrower tasks, improving accountability and throughput compared with unstructured self-coordination.

3. 3

   The system explicitly traded a small, stable error rate and later reconciliation for higher throughput, and the resulting harness provided useful linear scaling of token throughput with minimal overhead.