Scaling up your microservice testing with Apache Spark—Part 2

Summary

Part 2 describes how to build a repeatable Apache Spark replay harness for testing high-impact microservices against historical, production-shaped inputs without reimplementing the service. The design turns dependency state—such as database snapshots, versioned rules, merchant configuration, eligibility state, and logged service responses—into datasets, while representing side effects as output rows and matching fidelity to the decision under test. A thin Spark wrapper reconstructs ReplayInput objects, invokes the same deterministic decision engine used online, compares current and candidate results, and writes traceable diffs with intermediate values and rule explanations. The source cautions that incomplete context, distributed state, asynchronous workflows, and external actors limit replay’s predictive power, so it should model explicit assumptions rather than claim full production simulation. It presents migrations as a practical point to establish these boundaries and use replay to reduce risk.

Takeaways

1. 1

   Dependency state must be represented as replay datasets, including historical snapshots, versioned rules, effective-dated configuration, eligibility state, and logged requests and responses.

2. 2

   The Spark job should wrap shared production decision logic rather than reimplement business rules; otherwise, the replay can drift from production and create false confidence.

3. 3

   Replay outputs should include trace identifiers, current and candidate results, selected intermediate values, rule paths, and explanation fields so differences can be debugged and reviewed.