Loading post…
ThumbtackBrian TerczynskiProblem & solution
Headless Screenshot Testing in Thumbtack’s Android Apps
Summary
Thumbtack required a faster, emulator-free method to execute over 1,500 Android screenshot tests during continuous integration. Previously, running UI tests on physical devices or emulators proved too slow for code review verification jobs, causing tests to run post-merge and allowing regressions to persist. After evaluating Paparazzi and Google's experimental Compose Preview Testing, the team selected Roborazzi, which runs on top of Robolectric's native graphics engine within a standard JVM. This headless setup accommodates custom Activities, Dagger field injection, and legacy View systems while executing alongside standard unit tests. Moving screenshot tests to the unit test level enabled gating merges on passing tests, colocating tests inside feature modules, and increasing total CI job runtime by only twenty percent.
Context
Thumbtack maintained over 1,500 Android UI screenshot tests that originally required physical devices or emulators to execute drawing operations. Because emulator execution was too slow, these tests could not run during Jenkins code review verification jobs. Consequently, screenshot tests ran post-merge, leaving UI test failures unresolved for days in the main branch.
Approach / What changed
The team adopted Roborazzi, a headless screenshot testing library built on Robolectric's native graphics framework, enabling tests to run in a plain JVM alongside unit tests. They rejected Paparazzi because it lacked support for custom Activity classes required for Dagger field injection in legacy Views, and bypassed Compose Preview Testing due to bugs and incompatibility with legacy View System code.
Takeaways
Running screenshot tests in a plain JVM via Robolectric native graphics enabled over 1,500 UI tests to gate pull requests directly in Jenkins verification jobs.
Paparazzi was ruled out because Thumbtack's legacy code relied on Dagger field injection requiring custom Activity and Application classes.
Host platform rendering discrepancies between UNIX and MacOS in Robolectric native graphics were resolved by introducing a small pixel shift tolerance during image comparisons.
Source and freshness
- Source
- medium.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.