---
title: "Continually improving our agent harness"
description: "Cursor describes its agent harness as an iterative product process: vision-driven hypotheses are tested through offline evals, real-world A/B experiments, and instrumentation. Its context strategy has shifted from static codebase context and guardrails toward dynamic retrieval as models improve, while model-specific prompts and tools match training conventions. Quality measurement combines CursorBench with latency, token efficiency, tool-call errors, cache behavior, code Keep Rate, and language-model analysis of user responses. The team classifies expected and unknown tool errors, uses per-tool and per-model anomaly baselines, and automates log investigations and backlog tickets; one sprint reduced unexpected tool-call errors by an order of magnitude. The harness also supports model switching and subagents, although switching can cause cache misses and summary information loss, and the post argues that multi-agent orchestration will make harness engineering increasingly central."
---

# Continually improving our agent harness

[Cursor](https://yomu.fyi/company/cursor) · Stefan Heule & Jediah Katz · Apr 30, 2026

**Type:** Explainer

## Summary

Cursor describes its agent harness as an iterative product process: vision-driven hypotheses are tested through offline evals, real-world A/B experiments, and instrumentation. Its context strategy has shifted from static codebase context and guardrails toward dynamic retrieval as models improve, while model-specific prompts and tools match training conventions. Quality measurement combines CursorBench with latency, token efficiency, tool-call errors, cache behavior, code Keep Rate, and language-model analysis of user responses. The team classifies expected and unknown tool errors, uses per-tool and per-model anomaly baselines, and automates log investigations and backlog tickets; one sprint reduced unexpected tool-call errors by an order of magnitude. The harness also supports model switching and subagents, although switching can cause cache misses and summary information loss, and the post argues that multi-agent orchestration will make harness engineering increasingly central.

## Context

Cursor treats agent quality as difficult to measure and continually changing with model capabilities, harness complexity, tool failures, and model-specific behavior. The post also identifies challenges in preserving context and performance when users switch models mid-conversation.

## Approach / What changed

The harness combines dynamic context retrieval, model-specific prompts and tool formats, offline benchmarks, online A/B tests, quality and error metrics, anomaly detection, automated log investigation, and subagents. Model changes are evaluated and tuned through experiments, team usage, and targeted prompt or tool adjustments.

## Takeaways

- Code Keep Rate and language-model analysis of user responses complement operational metrics such as latency, token efficiency, tool-call count, and cache hit rate when assessing agent quality.
- Unknown tool errors are treated as harness bugs, while expected errors are classified by cause and monitored with per-tool, per-model anomaly baselines to detect regressions.
- Switching models mid-conversation changes prompts and tools, can cause provider- and model-specific cache misses, and may lose details when the conversation is summarized; subagents provide a fresh-context alternative.

**Tags:** [AI Agents](https://yomu.fyi/topic/ai-agents), [Cursor](https://yomu.fyi/topic/cursor), [Observability](https://yomu.fyi/topic/observability)

- Source: [Cursor](https://cursor.com/blog/continually-improving-agent-harness)
- Source URL: https://cursor.com/blog/continually-improving-agent-harness
- Ingested by Yomu: 2026-09-02T11:42:54.974Z

[Read original post](https://cursor.com/blog/continually-improving-agent-harness)