Go Modules- A Guide for monorepos (Part 1)

Summary

Grab transitioned its large Go monorepo dependency management from Glide to Go modules while retaining an existing vendor directory structure. The team generated root go.mod configurations from glide.yaml and used go mod vendor without directly enabling module-mode builds. Incompatible nested sub-vendor paths were excluded by placing empty go.mod files, relying on the rule that modules cannot contain other modules. Post-migration maintenance revealed challenges with dependency inheritance and implicit go.mod updates during builds, which engineers investigated using go mod graph and digraph to trace dependency paths.

Context

Managing dependencies in a large monorepo with multiple Glide lock files led to non-reproducible vendor directories, prompting Grab to adopt Go modules to align with official Go toolchains.

Approach / What changed

Grab scripted the generation of go.mod files from Glide manifests, managed the vendor folder with go mod vendor, isolated conflicting nested paths using empty go.mod files, and inspected unexpected dependency updates using go mod graph.

Takeaways

• A Go module cannot contain other modules, meaning Go ignores nested module directories when running commands at the monorepo root.
• Placing an empty go.mod file in problematic nested paths effectively excludes them from the root module's go mod vendor execution.
• Implicit updates to go.mod can occur automatically during standard commands like go build if broken imports or local replace directives exist.

Related reading

• Grab · Aug 12, 2020

  Go Modules- A Guide for monorepos (Part 2)

  Managing dependencies in a multi-module monorepo created developer friction at Grab due to unexpected changes from previous vendoring attempts and accidental imports. Because Go modules were not yet enabled directly for builds, the team implemented a continuous integration check that executes go mod vendor and rejects merge requests if any diffs exist in go.mod or the vendor directory. Adopting this CI check required configuring SSH deploy keys for private repositories, adding retry logic for network-related false positives, and standardizing on a single Go version to prevent checksum discrepancies. To streamline ongoing maintenance across hundreds of dependencies, the team developed an automated tool named AutoVend Bot. The bot runs go list -m -u all to detect updates and opens a scheduled batch of merge requests each day for human review.

  Michael Cartmell

• Grab · Jan 26, 2021

  Serving Driver-partners Data at Scale Using Mirror Cache

  Grab's Drivers Data service handles up to 10,000 requests per second during peak hours to supply driver information across backend microservices. The original setup used MySQL with Redis and standalone in-memory local caches, but yielded a low 25% local cache hit rate due to traffic patterns characterized by high burst frequency for individual drivers alongside redundant database calls across nodes. To solve this, the team developed Mirror Cache, an in-memory caching system that pairs Dgraph's Ristretto library with an asynchronous gRPC replication layer to mirror updates across cluster nodes. The replicator batches updates within the same AWS availability zone and forwards data to single nodes across zones to minimize transfer overhead. Production deployment increased the in-memory cache hit rate to approximately 75% and reduced direct MySQL queries by 5%.

  Indrajit Sarkar

• Grab · Jun 30, 2021

  Debugging High Latency Due to Context Leaks

  Market-Store, Grab's feature store for real-time machine learning features, experienced latency spikes from under 200 milliseconds to 2 seconds as traffic grew. Metrics and logs showed no direct correlation to API issues, but heap profiling with PPROF revealed continuously increasing memory held by child contexts. Further analysis tracked the leak to an update in Grab's open-source Async Library, which switched background contexts to uncancelled task contexts for worker runners. Because parent contexts maintained references to these uncancelled child contexts, the garbage collector could not reclaim their memory. This progressive memory exhaustion directly degraded API latency.

  Sourabh Suman

• Grab · Nov 23, 2020

  Democratising Fare Storage at Scale Using Event Sourcing

  Grab's legacy system stored booking and fare details in a single relational table, creating a bloated booking entity that tracked only the latest fare state and hindered rapid feature iteration. To resolve scalability, stability, and debugging challenges across millions of daily bookings, the team developed Fare Storage using the Event Sourcing pattern. The new architecture persists all fare modification events chronologically in DynamoDB, backed by a cache for eventually consistent reads and message streaming for downstream processing. The platform employs optimistic locking with versioning to manage concurrent updates, enforces idempotency through client-generated transaction UUIDs, and delegates metadata serialization to an SDK to prevent storage API changes.

  Sourabh Suman