Loading post…
Stack OverflowVarun JindalExplainer
Part 6: An operating system for coding agents: the disciplined build
Summary
Coding agent projects decay because agents rediscover files, contradict past decisions, and stop writing tests. The fix is a harness: a constitution of non-negotiable rules, an autonomy boundary that permits all reversible actions but gates irreversible ones, tiered context files per module, durable per-fact memory, a multi-lens review panel, and a post-edit hook that updates context in the same diff. On Terminal-Bench 2.0, changing only the harness lifted an agent from 52.8% to 66.5%, a 13.7-point gain with the same model. For multiple agents, coordination uses shared append-only logs, an ownership map by directory, claim/release protocols, and a decision log enforced during review. The model is about a tenth of long-term software quality; the harness is the rest and is buildable.
Context
This article is the final part of a six-part series on running LLM systems in production, focusing on the disciplined build layer that governs coding agents to maintain quality over long projects.
Approach / What changed
The harness consists of a constitution (non-negotiable rules read every session), an autonomy boundary implemented as an allow/ask/deny permission matrix, tiered context (root CONTEXT.md, per-module CONTEXT.md, DECISIONS.md), durable memory (per-fact files indexed by MEMORY.md plus a graph), a multi-lens review panel (product, architect, infra, security, test-eng sub-agents with confidence thresholds), and a post-edit hook that emits a checklist to update context/memory/graph in the same diff. For multiple agents, coordination uses shared append-only logs, an ownership map with claim/release protocols, branch-per-task with human merge, and a decision log enforced during review.
Takeaways
On Terminal-Bench 2.0, changing only the harness (system prompt, tools, middleware) lifted a coding agent from 52.8% to 66.5%, a 13.7-point gain, using the same model.
Define an autonomy boundary that permits all reversible actions (read, edit, write, git add/commit, test commands) automatically and gates irreversible ones (git push, deploy) behind human approval.
Multiple agents on the same repository coordinate through shared append-only logs, a directory-based ownership map with claim/release protocols, branch-per-task with human merge, and a decision log wired into review.
FAQ
What causes coding agent quality to decay over weeks?
Agents rediscover files, contradict past decisions, guess test commands, and stop writing tests because they lack a harness. The fix is not a better model but an operating system around the model that enforces rules and updates context consistently.
How should you set an agent's autonomy boundary?
Allow all reversible actions (read, edit, write, grep, glob, git add/commit, npm test/run/ci, pytest) automatically. Gate irreversible actions (git push, npm publish, aws, terraform apply, kubectl apply) behind human approval. Deny dangerous operations like force push.
How do multiple agents coordinate without conflict?
They share state through an ownership map defaulted by directory, a claim/release log, branch-per-task with human merge, and an append-only decision log enforced during review. They do not message each other directly.
Source and freshness
- Source
- stackoverflow.blog
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.