Tackling UI Test Execution Time Imbalance for Xcode Parallel Testing

Summary

Parallel test execution in Xcode can suffer from test time imbalance when tasks finish at significantly different times across parallel simulator workers. Analysis of Xcode scheduling logs shows that the runner groups tests by test class and dispatches all tests from the same class to a single simulator. Attempts to customize the suite by swizzling XCTestSuite fail because made-up suites initialize only after tests are dispatched. To overcome this grouping constraint, unique tokens or test names are appended to the class name component in `-only-testing` command-line arguments. This trick forces Xcode to treat each test as an independent class, successfully distributing individual tests across separate workers.

Takeaways

1. 1

   Test time imbalance is measured as the difference between the maximum and minimum worker finish times across parallel execution instances.

2. 2

   Swizzling XCTestSuite methods to split suites cannot bypass worker grouping because custom test suites initialize only after Xcode dispatches tests to workers.

3. 3

   Appending unique strings to class names within `-only-testing` arguments causes Xcode to treat every method as a separate class, enabling single-test dispatching to free workers.