Tackling UI Test Execution Time Imbalance for Xcode Parallel Testing

Summary

Parallel test execution in Xcode can suffer from test time imbalance when tasks finish at significantly different times across parallel simulator workers. Analysis of Xcode scheduling logs shows that the runner groups tests by test class and dispatches all tests from the same class to a single simulator. Attempts to customize the suite by swizzling XCTestSuite fail because made-up suites initialize only after tests are dispatched. To overcome this grouping constraint, unique tokens or test names are appended to the class name component in `-only-testing` command-line arguments. This trick forces Xcode to treat each test as an independent class, successfully distributing individual tests across separate workers.

Context

Xcode parallel testing groups tests by class and assigns all methods in a class to the same simulator worker, causing test execution time imbalance across parallel runners in CI pipelines.

Approach / What changed

Modifying the test identifiers passed to the `-only-testing` argument of `xcodebuild` by inserting unique tokens or method names into the class name segment to prevent Xcode from grouping tests together.

Takeaways

• Test time imbalance is measured as the difference between the maximum and minimum worker finish times across parallel execution instances.
• Swizzling XCTestSuite methods to split suites cannot bypass worker grouping because custom test suites initialize only after Xcode dispatches tests to workers.
• Appending unique strings to class names within `-only-testing` arguments causes Xcode to treat every method as a separate class, enabling single-test dispatching to free workers.

Related reading

• Grab · Apr 19, 2021

  How We Improved Agent Chat Efficiency with Machine Learning

  Agent typing time represented a large portion of Grab's chat support journey, and 85% of messages were still free typed because agents customized static templates to fit their personal style. To accelerate typing across multilingual markets without robotic templates, Grab built SmartChat, a machine learning feature that provides contextual sentence completion. The team opted for a lightweight seq2seq architecture using single-layered GRU encoder-decoders in TensorFlow instead of bulky attention models to keep model latency under 100ms. The user interface was implemented in React using a content-editable div with inline typeahead suggestions activated via keyboard shortcuts.

  Suman Anand

• Grab · Jul 29, 2021

  How We Cut GrabFood.com’s Page JavaScript Asset Sizes by 3x

  GrabFood.com faced high cloud costs while serving over 1 TB of weekly network egress across 175 million requests. To minimize egress and improve page performance, the engineering team audited and reduced their webpack dependencies using tools such as webpack-bundle-analyzer and dependency-cruiser. Their strategies included lazy loading rarely used libraries, unifying duplicate modules under single entry points, and substituting utility libraries like axios with native browser Web APIs. Additionally, altering implementation approaches, such as replacing client-side signed JWT cookie encoding with plain JSON strings, eliminated heavy Node dependencies like crypto. These optimizations reduced JavaScript static assets from 750 KB to 250 KB, decreased CloudFront costs by 20%, and accelerated build times by 3.6x.

  Gibson Cheng

• Grab · Jul 13, 2021

  App Modularisation at Scale

  Grab transitioned its monolithic mobile application into a modular architecture to resolve increasing code conflicts, slow releases, and difficult team collaboration. The team decomposed the single module by establishing base infrastructure modules, shared UI and utility libraries, discrete feature modules, and bridge kit modules for inter-module communication. Dependency injection using Dagger ties these components together in the main app module while preventing feature modules from directly depending on one another. The architecture spans over 1,000 modules across the app, with more than 200 modules in the Grab Financial Group payments domain where over 95% of modules build in under 15 seconds. This approach accelerated Gradle CI and local builds through parallel compilation and caching, though it increased Gradle sync times, IDE memory usage, and configuration maintenance overhead.

  Amar Jain

• Grab · Dec 3, 2020

  How Grab is Blazing Through the Superapp Bazel Migration

  Grab's mobile superapp scaled past 2.5 million lines of code across both Android and iOS, leading to unsustainable local and CI build times under Gradle and Xcode. To address these bottlenecks, the engineering team analyzed their dependency trees and introduced an internal tool to calculate and optimize the build critical path. They also deployed a Kubernetes-autoscaled remote build system using Mainframer for Android and implemented Test Impact Analysis to run only affected tests in pre-merge validation. While dependency decoupling yielded modest 7% to 10% gains and iOS remote builds proved unscalable on Apple hardware, Android remote builds reduced local compile times by up to 50%, and targeted test execution reduced pre-merge pipeline durations by more than 30%.

  Sergii Grechukha