---
title: "Modular regression testing framework built on AWS for large-scale systems"
description: "A modular regression testing framework built on AWS replays a sample of production traffic against new code and compares results to validate no behavior changes before deployment. It uses Step Functions for orchestration, Lambda for replay, DynamoDB for capturing requests and responses, and CloudWatch for logging and alarming on match fraction. The design is minimally invasive, requiring no changes to stateless systems and only a side-effect skip for stateful ones, and can be inserted at API, service, database, or message queue boundaries. In over six months of production use, the framework caught five regressions with zero false positives, using a 99.9% match fraction alarm to block problematic deployments via a deployment gate. The post covers three use cases—refactoring, feature toggle deployments, and impact boundary validation—and provides code examples for capture, replay, and evaluation functions."
---

# Modular regression testing framework built on AWS for large-scale systems

[AWS](https://yomu.fyi/company/aws) · Hannah Zhang · Nov 12, 2026

**Type:** Tutorial

## Summary

A modular regression testing framework built on AWS replays a sample of production traffic against new code and compares results to validate no behavior changes before deployment. It uses Step Functions for orchestration, Lambda for replay, DynamoDB for capturing requests and responses, and CloudWatch for logging and alarming on match fraction. The design is minimally invasive, requiring no changes to stateless systems and only a side-effect skip for stateful ones, and can be inserted at API, service, database, or message queue boundaries. In over six months of production use, the framework caught five regressions with zero false positives, using a 99.9% match fraction alarm to block problematic deployments via a deployment gate. The post covers three use cases—refactoring, feature toggle deployments, and impact boundary validation—and provides code examples for capture, replay, and evaluation functions.

## Context

AWS blog post by an engineer detailing a modular regression testing framework using AWS services for large-scale systems.

## Approach / What changed

The framework captures production requests and responses to DynamoDB, replays them via Lambda against new code in a pre-production environment, and compares results. It uses Step Functions to orchestrate start/stop, SSM to store simulation state, and CloudWatch alarms on match fraction to gate deployments.

## Takeaways

- The framework replays production traffic, not synthetic data, making regression detection accurate.
- It is minimally invasive: stateless systems need no changes; stateful systems add a check to skip side effects.
- In production, it caught five regressions with zero false positives over six months.

## FAQ

### How does the framework handle stateful systems?

Stateful systems add a check that recognizes replayed requests and skips side effects, preventing duplicate records or notifications.

### What match fraction threshold is used to block deployments?

Teams commonly set alarms at 99.9% match fraction to block potentially problematic deployments.

### What types of systems work best with this framework?

Computation-heavy, stateless systems like pricing calculators, validation engines, and data transformation pipelines work best.

**Tags:** [AWS](https://yomu.fyi/topic/aws), [CI/CD](https://yomu.fyi/topic/ci-cd), [Regression Testing](https://yomu.fyi/topic/regression-testing), [Simulation](https://yomu.fyi/topic/simulation)

- Source: [AWS](https://aws.amazon.com/blogs/architecture/modular-regression-testing-framework-built-on-aws-for-large-scale-systems)
- Source URL: https://aws.amazon.com/blogs/architecture/modular-regression-testing-framework-built-on-aws-for-large-scale-systems
- Ingested by Yomu: 2026-11-13T00:00:14.238Z

[Read original post](https://aws.amazon.com/blogs/architecture/modular-regression-testing-framework-built-on-aws-for-large-scale-systems)
