Loki, a Dynamic Mock Server for HTTP/TCP Testing

Summary

Grab built Loki, a dynamic mock server written in Golang that simulates backend services on local developer machines and CI pipelines. Mobile app testing previously suffered from heavy dependencies on complex, brittle staging environments and interconnected services communicating over HTTP, HTTPS, and TCP. Loki handles both HTTP and TCP traffic on distinct ports while exposing a unified RESTful API to manage test expectations. It provides runtime flexibility through sandboxed JavaScript execution, configurable request sequence ordering, and an in-memory cron scheduler for TCP push messages. Adopting Loki decoupled mobile releases from staging stability, improving delivery cycles and enabling automated UI testing with Espresso and XCUITest.

Context

End-to-end mobile app testing at Grab faced significant friction due to heavy dependencies on backend services, staging downtime, data corruption, and complex multi-app business flows spanning HTTP and TCP.

Approach / What changed

Grab developed Loki, a containerized Golang mock server with pluggable storage backends (MySQL, Redis, or in-memory cache) that supports dynamic JavaScript injection, sequenced HTTP responses, and an in-memory cron system to push scheduled TCP events.

Takeaways

• Mocking TCP required handling long-running client-server connections, configurable request delimiters for JSON deserialization, and unsolicited push messages via an in-memory cron scheduler.
• Dynamic responses can be computed at runtime using an in-house JavaScript execution library running inside an isolated sandbox.
• The httpTimesAndOrder API enables developers to configure ordered response state transitions on the same endpoint to simulate sequential lifecycle changes like ride booking stages.

Related reading

• Grab · Sep 18, 2018

  Mockers - Overcoming Testing Challenges at Grab

  Grab operates over 250 microservices communicating over HTTP and gRPC, making shared staging environments costly, ambiguous in ownership, and fragile due to inconsistent data and uncoordinated deployments. To address these testing bottlenecks, Grab created Mockers, a Go SDK and CLI tool backed by a central monorepo of mock servers for local-box and CI testing. Mockers automatically generates HTTP and gRPC mock servers from Swagger specifications and protobuf files, returning configured network responses without internal business logic. By incorporating Grab's in-house chaos SDK middleware, Mockers also enables repeatable resiliency and contract testing locally without relying on code-level mocks. While Grab still mandates integration testing on distributed staging environments with live data, Mockers enables developers to detect complex defects and contract mismatches earlier.

  Mayank Gupta

• Grab · Dec 23, 2019

  Marionette - Enabling E2E User-scenario Simulation

  Conducting end-to-end testing across Grab's transport microservices became difficult due to service availability, environment construction, cross-service authentication, and complex data setups for real-world user accounts. To address these challenges without relying on physical mobile devices or emulators, Grab built Marionette, an internal simulation platform for passenger and driver interactions. The platform provisions required test data, coordinates booking lifecycles, and isolates test executions across distinct user groups using localized cohorts. Engineers can configure driver and passenger behaviors, execute workflows, and run load or integration tests through a dedicated user interface, a Go SDK, and RESTful APIs.

  Anish Jha

• Grab · Apr 18, 2017

  Deep Dive into iOS Automation at Grab - Integration Testing

  Grab replaced third-party cloud CI services with an in-house Xcode Server pipeline to run automated iOS integration tests across an expanding engineering team. Instead of performing fresh repository clones on shared cloud instances, Xcode Server pulls code incrementally and mirrors local developer configurations. The team automated testing bots through Ruby scripts and Arcanist hooks tied to Phabricator code reviews, dynamically provisioning bots on diffs and deleting them after merges. This architecture preserves step-by-step UI test execution logs and screenshots for easier failure diagnostics. To maintain stability on on-premises hardware, pre-integration scripts clear application cache states using simctl commands.

  Sun Xiangxin

• Grab · Mar 31, 2023

  Evolution of quality at Grab

  As Grab expanded its superapp, manual QA sign-offs led to prolonged testing cycles and a peak in major production defects around 2019. To counter defect leakage and improve delivery pace, the engineering organization transitioned to a shift-left testing strategy across its software development lifecycle. Quality engineers began writing Given/When/Then acceptance tests prior to coding so that software engineers could execute them during development, accompanied by Definition of Ready and Definition of Done standards. The team also structured a multi-layer test strategy spanning unit tests, pipeline-executed UI component tests with mocked APIs, backend integration tests, and a custom internal end-to-end framework. Between 2019 and 2022, these changes reduced major and critical production issues by 60% and development-phase critical bugs by 40%.

  Abby Alcantara