---
title: "Testing"
description: "271 posts about Testing, summarised, each linking to the original."
---

# Testing
> 271 posts about Testing, summarised, each linking to the original.

## Articles

### [Upgrading Kickstarter to Rails 5](https://yomu.fyi/post/upgrading-kickstarter-to-rails-5.md)
- Company: [Kickstarter](https://yomu.fyi/company/kickstarter.md)
- Author: Logan McDonald
- Published: Nov 6, 2017
- Type: Problem & solution

Kickstarter upgraded its main Rails application to Rails 5, alongside payments app Rosie, to support site security and performance while moving a nine-year-old legacy codebase forward. The team planned a three-month project, upgraded Ruby 2.2.5 to 2.4.x, classified release-note changes, resolved dependency conflicts, used Rails’ upgrade script, and moved through twelve documented steps. Work included merging Rails 4.2-compatible fixes, repairing nearly 4,000 CI test failures from a roughly 70 percent coverage baseline, removing deprecations, smoke-testing major flows, backporting more than thirty pull requests, and preparing reviewers and deployment contingencies. The upgrade met its timeline without rollbacks or major downtime, although untested bugs increased Zendesk tickets and exposed issues involving ActionController::Parameters, tracking events, and mobile applications. A retrospective emphasized planning, documentation, cross-team communication, and broader native-team participation for future upgrades.


### [Evolution of the Heroku CLI: 2008-2017](https://yomu.fyi/post/evolution-of-the-heroku-cli-2008-2017.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Jeff Dickey
- Published: Aug 15, 2017
- Type: Explainer

Between 2008 and 2017, Heroku's CLI evolved from Ruby (v1–v3) through Go/Node versions v4 and v5 to a pure Node v6 implementation. The rewrite addressed slower Windows performance, separation of the CLI's runtime from users' Ruby environments, migration to API v3, and limitations in Ruby-based plugin dependencies. V4 kept existing command syntax and output while adding features such as non-TTY color suppression and --json output; its Go core used Node for plugins, and the CLI became a collection of core plugins that could also be developed and installed independently. The team migrated commands incrementally, then added Flow, Babel, and improved tests, including Jest tests that run the flag parser. The resulting foundation supports modular, tested feature development, with cli-engine and more interactive interfaces planned.


### [Come and #hackallthethings at Grab](https://yomu.fyi/post/come-and-hackallthethings-at-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Grab Engineering
- Published: Jul 11, 2017
- Type: Announcement

Grab has officially launched a public bug bounty program in partnership with HackerOne to strengthen the security of its platform. This rollout follows a private bounty initiative operated over the previous year, during which the organization worked with over 350 security researchers and resolved nearly 200 awarded bug reports. The new public program invites external researchers to scrutinize Grab's code for critical flaws, including remote code execution, SQL injections, and exportable cross-site scripting vulnerabilities. To support ethical and responsible disclosure, Grab offers payouts reaching up to $10,000 per valid vulnerability report based on severity and impact.


### [Functional Swift Conference 2017 — Brooklyn](https://yomu.fyi/post/functional-swift-conference-2017-brooklyn.md)
- Company: [Kickstarter](https://yomu.fyi/company/kickstarter.md)
- Author: Brandon Williams
- Published: Apr 28, 2017
- Type: Announcement

Kickstarter hosted the fourth Functional Swift Conference on April 15th, marking the third time the event had been held at its office. The all-day gathering brought together 10 speakers, including four Kickstarter engineers, and 90 attendees to discuss what functional programming can contribute to better Swift code. Sessions covered recursive enums and KD-trees, testing side-effects, domain-specific languages with GraphQL, algebraic laws, monoids, predicates, sorting functions, applicative functors, parsing, and type inference. Examples included using value types and recursive data structures to describe complicated objects and live-coding an applicative parsing API that generates its own documentation. The closing type-system talk examined why inference is hard and described the Swift compiler team’s use of graph theory to address inference problems practically.


### [Deep Dive into iOS Automation at Grab - Integration Testing](https://yomu.fyi/post/deep-dive-into-ios-automation-at-grab-integration-testing.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Sun Xiangxin
- Published: Apr 18, 2017
- Type: Problem & solution

Grab replaced third-party cloud CI services with an in-house Xcode Server pipeline to run automated iOS integration tests across an expanding engineering team. Instead of performing fresh repository clones on shared cloud instances, Xcode Server pulls code incrementally and mirrors local developer configurations. The team automated testing bots through Ruby scripts and Arcanist hooks tied to Phabricator code reviews, dynamically provisioning bots on diffs and deleting them after merges. This architecture preserves step-by-step UI test execution logs and screenshots for easier failure diagnostics. To maintain stability on on-premises hardware, pre-integration scripts clear application cache states using simctl commands.


### [Why you should co-locate your Xcode tests](https://yomu.fyi/post/why-you-should-co-locate-your-xcode-tests.md)
- Company: [Kickstarter](https://yomu.fyi/company/kickstarter.md)
- Author: Brandon Williams
- Published: Mar 23, 2017
- Type: Problem & solution

Xcode projects with unit tests commonly separate implementation and test files into two directories, often mirroring the source structure in the tests directory. The post argues that this arrangement creates maintenance overhead, slows navigation, obscures how well-tested a codebase is when browsed or reviewed, and can leave tests with lower coding standards. It proposes co-locating each unit-test file beside its implementation file while retaining the test target association, accepting more files per directory in exchange for clearer relationships and easier movement between code and tests. The same SwiftLint configuration is used for implementation and test code, treating tests as first-class code. Tests without a single-file correspondence, including integration or UI tests, should remain in separate directories and may warrant separate targets; the authors say their codebase has adopted the style.


### [This Rocket Ain't Stopping - Achieving Zero Downtime for Rails to Golang API Migration](https://yomu.fyi/post/this-rocket-ain-t-stopping-achieving-zero-downtime-for-rails-to-golang.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Lian Yuanlin
- Published: Oct 18, 2016
- Type: Problem & solution

Grab transitioned its public passenger app APIs from a legacy Rails application to a Golang service-oriented architecture to consolidate its codebase and engineering teams. Initial attempts to proxy traffic through a cloned Rails server via gRPC were abandoned after encountering TCP load imbalances during autoscaling events and memory leaks in the gRPC Ruby gem. The team pivoted to direct logic migration, porting Ruby logic directly into Go while decomposing modules into standalone services. Verification relied on log-based load testing and live shadow testing, where write operations were safely validated using mock data access layers that evaluated expected database outcomes. Production rollout progressed endpoint-by-endpoint using requests-per-second traffic throttling and prewarmed AWS Elastic Load Balancers before executing the final DNS switch.


### [Grab Vietnam Careers Week](https://yomu.fyi/post/grab-vietnam-careers-week.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Grab Engineering
- Published: Oct 14, 2016
- Type: Announcement

Grab announced its first Grab Vietnam Careers Week, held in Ho Chi Minh City from 22 to 26 October 2016, aimed at hiring engineers to improve regional transportation and reduce congestion. At the time, Grab operated on 23 million mobile devices supported by 460,000 drivers across Southeast Asia. The announcement features an interview with Singapore-based iOS engineer Hai Pham and Android engineer Son Nguyen discussing team culture, regular hackathons termed "Grabathons", and relocation to Singapore. Technical candidates undergo a Codility test and are evaluated on clean mobile architecture, testing, and performance maintenance. The engineers also offer advice on apartment hunting through online community forums and adapting to life in Singapore.


### [Why Test the Design with Only 5 Users](https://yomu.fyi/post/why-test-the-design-with-only-5-users.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Avinash Papatla
- Published: Aug 26, 2016
- Type: Explainer

Stakeholders often question whether qualitative testing with only five participants yields reliable design findings. The chosen research problem dictates the methodology, with qualitative usability evaluations requiring smaller cohorts than broad quantitative surveys. Jakob Nielsen's 1993 analysis demonstrated that testing with five participants uncovers 75 to 80 percent of usability issues, after which additional users yield diminishing returns. In agile environments, teams maximize efficiency by iterating designs across successive rounds of five participants rather than testing large cohorts simultaneously. This five-user threshold applies strictly to qualitative usability testing for specific feature flows rather than opinion gathering, A/B testing, or multi-segment user analysis.


### [Incident Preparation: Simulating Third-Party Downtime](https://yomu.fyi/post/incident-preparation-simulating-third-party-downtime.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Damien Mathieu
- Published: Mar 1, 2016
- Type: Tutorial

Heroku’s help application depends on the Platform API for authentication, authorization, and user data, so it must remain available to agents and customers during platform incidents. The open-sourced api-maintenance-sim app tests this requirement by imitating an outage in which every request receives an HTTP 503 Service Unavailable response. After deploying the simulator, teams point the application under test to its hostname through PLATFORM\_API\_URL, passed to platform-api’s connect\_oauth method; when unset, the gem uses the production URI. A heroku config:set command enables the alternate endpoint, making regular simulations possible and helping reveal what failure means for the application or service when the Platform API is unavailable, supporting preparation for incidents and communication during outages.


### [How We Migrated to Active Record 4](https://yomu.fyi/post/how-we-migrated-to-active-record-4.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Owen Ou
- Published: Nov 4, 2015
- Type: Problem & solution

Heroku Platform API had remained on Active Record 2.3.18, released in March 2013, creating maintenance and security burdens as support ended and custom query infrastructure replaced newer features. The team moved first to Active Record 3.0.20, preserving 2.x APIs, then directly to 4.2.4, using a feature branch, console-driven fixes, automated and acceptance tests, private deployments, staging, and a partial production rollout. The first step took one engineer about two weeks, while the second took two engineers about two weeks; deployment had no downtime but briefly required a rollback for an acts\_as\_paranoid bug. Performance improved 18% from 2.3.18 to 3.0.20 but worsened 13% from 3.0.20 to 4.2.4, challenging micro-benchmark expectations. The upgrade ultimately left performance at the same overall level, removed patches, restored supported status, and made future updates and training easier.


[Newer posts](https://yomu.fyi/topic/testing/page/13.md)