One Small Step Closer to Containerising Service Binaries

Summary

Engineering teams at Grab initiated a transition to containerized microservices to standardize environments, enhance security, and decouple services from internal runtime tooling. During this migration, developers noticed that statically-linked Go service binaries were reaching bloated sizes over 100 MB. By analyzing the binaries using the open-source tool go-binsize-viz alongside the Go nm toolchain, the team visualized compiled symbols as interactive treemaps. This analysis revealed that 11 MB of unused message format symbols were being pulled in because a generic interface shared a directory with auto-generated streaming code. Restructuring the packages to isolate interfaces from generated code successfully decreased the binary size down to 78 MB.

Context

During Grab's migration to a containerized microservice architecture, engineers noticed that statically-linked Go binaries were unexpectedly bloated to sizes over 100 MB, increasing container image size and deployment overhead.

Approach / What changed

Engineers used the go-binsize-viz utility and Go's nm tool to generate treemap visualizations of binary symbols, pinpointed unused transitive dependencies caused by auto-generated stream message formats sharing a package directory with an interface, and restructured the code to decouple them.

Takeaways

• Go imports all symbols defined within an imported directory and its transitive dependencies, rather than only the specifically referenced symbols.
• Placing a shared interface in the same directory as auto-generated streaming models caused services to needlessly compile 11 MB of unused message symbols into their binaries.
• Restructuring the flat package layout to separate generic interfaces from generated models reduced the target binary size from over 100 MB down to 78 MB.

Related reading

• Grab · Jan 8, 2020

  How We Prevented App Performance Degradation from Sudden Ride Demand Spikes

  Grab experienced severe system strain when sudden localized spikes in ride demand, triggered by events like heavy rain or concert dismissals, coincided with driver shortages. These localized bursts overloaded the platform and degraded the experience for users outside the affected areas. To mitigate this, engineers created the Spampede filter, a circuit-breaker mechanism placed at the start of the booking pipeline. The filter converts pickup locations into Geohash Integer buckets and partitions time using Unix timestamps, tracking unfulfilled requests in Redis with atomic increments and time-to-live expirations. When unallocated requests exceed configured thresholds within a specific bucket, the system immediately short-circuits new incoming bookings to protect overall platform stability.

  Corey Scott

• Grab · Mar 1, 2022

  Abacus - Issuing points for multiple sources

  Grab needed a centralised points management architecture to issue loyalty points across a growing catalog of products, membership tiers, and external partner exchanges. To address this, the engineering team built Abacus, an issuance platform designed to process millions of daily transactions with high availability. The system ingests completed transaction streams or API calls, dynamically computes points via configured multipliers, and passes calculations through Amazon Simple Queue Service queues. Once the Point Awarding module updates a persistent ledger, Abacus notifies consumers, emits events to Kafka for downstream consumers, and recalculates rolling point expiration dates.

  Chandrakanth

• Grab · Sep 23, 2020

  Our Journey to Continuous Delivery at Grab (Part 1)

  Around the end of 2018, Grab's backend architecture consisted of roughly 270 services managed through fragmented, manual deployment workflows. Engineers copied release parameters between build logs, wiki pages, Slack bots, and multiple Jenkins jobs, leading to high operational friction and an average of 10 business days between production updates for a service. To streamline delivery, Grab built Conveyor, an internal automation platform built on top of open-source Spinnaker. Conveyor introduced a custom user interface focused on pipeline visibility and a pipeline-as-code DSL called Artificer using Jsonnet files in the monorepository. The platform automatically registers build artifacts with commit metadata to eliminate manual parameter entry and automatically provisions integration, staging, and production pipelines.

  Sylvain Bougerel

• Grab · Jun 29, 2020

  How We Built Our In-house Chat Platform for the Web

  Grab extended its in-house chat platform to the web to support its internal Customer Support portal. Because the existing TCP gateway only supports unicast connections with one active connection per user, opening multiple browser tabs would repeatedly disconnect previous tabs. Rather than undertaking a complex migration to multicast connections on the server, the team adopted a hybrid client-side strategy using SharedWorker and BroadcastChannel APIs. The implementation uses a SharedWorker to maintain a single WebSocket connection per domain while a BroadcastChannel syncs events across all open tabs. A custom wrapper over the worker manages version transitions during deployments to avoid race conditions across tabs.

  Vasu Krishnamoorthy