Leveraging AI to build a faster iOS CI system

Summary

iOS continuous integration at Thumbtack previously required 30 to 40 minutes per pull request because every change triggered all 400 or more tests for an affected app. To solve node exhaustion and prepare for increased AI-generated code volume, the infrastructure team built Smart Test Selection. The system relies on a nightly pipeline that executes the full test suite with coverage instrumentation to generate a JSON map connecting source files to test classes. When a pull request opens, the pipeline looks up changed files in this map and runs only the relevant tests instead of the full suite. Built using Groovy, Ruby, and Bash, the tool reduces typical continuous integration time by approximately 50 percent while cutting documentation-only validation by about 90 percent.

Context

Thumbtack experienced 30 to 40 minute iOS continuous integration times, with peak queues exceeding 60 minutes. Running full test suites on every pull request became unsustainable due to rising code size, limited CI nodes, and an anticipated influx of AI-generated pull requests and tests.

Approach / What changed

The team implemented Smart Test Selection by running nightly instrumented test suites to generate a source-to-test JSON coverage map. Pull requests query this artifact to execute only relevant tests, handle edge cases like documentation and build configurations, and fall back to full suites when needed.

Takeaways

• Using runtime coverage rather than static analysis captures transitive dependencies automatically across dynamic dispatches, closures, and protocol conformances.
• Smart Test Selection cuts typical iOS continuous integration time by about 50 percent, documentation changes by roughly 90 percent, and build config changes by 65 percent.
• Special cases required custom handling, including a dedicated pbxproj diff analyzer for Xcode project files and image-naming convention mapping for snapshot tests.

Related reading

• Grab · Jun 22, 2026

  Scaling out Distroless adoption With AI

  Grab is transitioning its microservices to Distroless base images to eliminate unnecessary binaries and reduce vulnerability risks, but the migration risks runtime failures from missing shared objects and system utilities. To safely validate container execution in continuous integration without staging dependencies, the team relied on medium tests that run containerized services alongside internal dependencies managed by Testcontainers. Because hundreds of services lacked this test harness, Grab implemented an agentic workflow using Claude Code and Model Context Protocol integrations to inspect repositories, generate test boilerplate, and resolve configuration errors. Once test baselines are established, an automated patch-test-compare pipeline updates Dockerfiles, constructs multi-stage builds for necessary dynamic libraries, and creates draft merge requests for human approval.

  Jia Yee Chong

• Thumbtack · May 22, 2026

  AI as the Next Abstraction Layer: How I see engineering evolving at Thumbtack