# Your agent should understand what you see

[Sentry](https://yomu.fyi/company/sentry) · Mihir Mavalankar · Jul 16, 2026

**Type:** Problem & solution

## Summary

Sentry's Seer Agent initially parsed user screens using DOM-scraped character-by-character ASCII grids, leading to severe token consumption, model context rot, and an inability to support interactive actions. To resolve these bottlenecks, Sentry replaced visual text scraping with a composable semantic context system where React UI components directly declare their state, metadata, and interactive affordances. A root React context provider maintains a registry of component nodes organized hierarchically through higher-order wrappers, serializing the resulting structure to JSON and backend-generated Markdown within the system prompt. Across twenty-four live pages, this approach reduced dashboard context tokens from roughly 5,500 to 1,300 while maintaining identical satisfaction rates and tool-call volumes across more than 14,000 evaluated conversations.

## Context

Sentry's Seer Agent originally gathered page context by scraping the DOM into an ASCII character grid. Single dashboards reached p95 token measurements of 50,000 tokens or more, consuming 85-93% of the system prompt on average. This high volume caused context rot in Claude Sonnet models, degraded multi-turn conversation quality, and provided no structured way to express interactive elements or execute UI actions.

## Approach / What changed

Sentry implemented a bottom-up semantic context system using React. A root context provider maintains a flat registry of nodes, populated when components mount via a higher-order component wrapper. Components supply their own state, modes, and filters through hooks, deriving hierarchy directly from the React component tree. The registry is assembled into a nested tree, serialized to JSON, and converted to Markdown on the backend for inclusion in the system prompt.

## Takeaways

- Switching from ASCII snapshots to semantic context dropped page context prompt usage from 85–93% down to 50–80%, reducing dashboard tokens from ~5,500 to ~1,300 and issue detail tokens from ~2,100 to ~300.
- Across over 14,000 conversation runs evaluated by an LLM as a judge, the structured context approach matched the prior ASCII snapshot approach in both user satisfaction rates and average tool calls per conversation.
- The composable architecture derives component hierarchy automatically from React's component tree, allowing unmigrated UI components to fall back seamlessly to ASCII snapshots.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [LLMs](https://yomu.fyi/topic/llm), [Observability](https://yomu.fyi/topic/observability)

[Read original post](https://blog.sentry.io/seer-agent-page-context)