# Agentic Testing: Where Agents Fit in the E2E Testing Stack

[Slack](https://yomu.fyi/company/slack) · Sergii Gorbachov · Jun 11, 2026

**Type:** Benchmark

## Summary

Traditional end-to-end tests validate rigid user journeys, whereas agentic tests verify whether broad goals can be achieved by adapting actions dynamically. To evaluate agentic testing tradeoffs, researchers executed over 200 runs across Playwright Model Context Protocol (MCP), Playwright CLI, and agent-generated Playwright tests using Claude models. Playwright MCP demonstrated high reliability with failure rates of 0% on simple thread replies and approximately 12% on complex search discovery flows. Playwright CLI and generated code struggled more on complex workflows, exhibiting failure rates of approximately 20% and 48% respectively. Although generated tests were faster with average runtimes of roughly three minutes, agentic testing provides a distinct exploratory layer atop deterministic CI test suites.

## Context

Traditional end-to-end testing enforces specific deterministic UI paths, making it unclear whether high-cost, multi-minute agentic tests that verify goals dynamically can practically fit into modern testing stacks.

## Approach / What changed

Ran over 200 automated executions across simple and complex workflows to compare three execution models: an agent using Playwright MCP, an agent using Playwright CLI, and agent-generated Playwright code using Claude models.

## Takeaways

- Playwright MCP achieved failure rates of 0% on simple workflows and roughly 12% on complex workflows, outperforming Playwright CLI which failed 12% to 20% of the time due to execution and session instability.
- Generated Playwright tests executed fastest with an average duration of approximately 3 minutes, but their failure rate spiked to around 48% on complex workflows due to UI state variability and abstraction mismatches.
- Agentic testing functions as an exploratory and debugging layer atop the testing pyramid rather than replacing fast, low-cost deterministic end-to-end tests in CI.

**Tags:** [Developer Experience](https://yomu.fyi/topic/developer-experience), [LLMs](https://yomu.fyi/topic/llm), [Testing](https://yomu.fyi/topic/testing)

[Read original post](https://slack.engineering/agentic-testing-where-agents-fit-in-the-e2e-testing-stack)