---
title: "Go"
description: "90 posts about Go, summarised, each linking to the original."
---

# Go
> 90 posts about Go, summarised, each linking to the original.

## Articles

### [Simpler Fly deployments for NodeJS, Rails, Go, and Java](https://yomu.fyi/post/simpler-fly-deployments-for-nodejs-rails-go-and-java.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Feb 10, 2020
- Type: Announcement

flyctl now supports building and deploying NodeJS, Rails, Go, and Java applications to Fly with two commands. The feature uses Cloud Native Buildpacks from buildpacks.io, turning an application into a Docker image during deployment without requiring users to understand Buildpacks. Cloud Native Buildpacks are described as an evolution of Heroku and Cloud Foundry systems, but are not compatible with their older buildpacks. After installing the latest flyctl, users can initialize an app with flyctl init --builder cloudfoundry/cnb:bionic and run flyctl deploy, then open it with flyctl open. Buildpack support complements Dockerfile-based image builds, while listed builders provide different language coverage and image sizes; the post recommends heroku/buildpacks:18 for its wider coverage.


### [Command Lines, Flyctl and Fly](https://yomu.fyi/post/command-lines-flyctl-and-fly.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Jan 29, 2020
- Type: Announcement

Fly presents the Fly Global Application Platform as a command-line-first service and introduces flyctl as the way to sign up for free and manage an application's entire lifecycle. The post argues that command-line interfaces integrate directly with modern workflows, are simple and expressive, and keep developers in their terminal and current directory. To replace its Node-based command-line technology, Fly built flyctl in Go as self-contained binaries for macOS, Linux, and Windows, and adopted Cobra for familiar command/subcommand behavior and contextual help. Web interfaces remain intended for rich dashboards and metric displays, alongside interactive terminal use. The command-line approach also makes instructions self-contained, scriptable, and repeatable, which the post identifies as valuable for support and operations, while the open GitHub repository invites feedback.


### [Plumbing At Scale](https://yomu.fyi/post/plumbing-at-scale.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Karan Kamath
- Published: Jan 6, 2020
- Type: Problem & solution

Grab's backend services process terabytes of data ingress per hour, generating recurring needs for stream transformations, joins, and time-windowed aggregations across diverse workloads. To support these asynchronous processing patterns across their Go ecosystem, the Coban team developed a managed, NoOps event sourcing and stream processing platform. The architecture packages stateless processing pipelines as Kubernetes deployments on AWS, polling Kafka event logs and using ScyllaDB as a shared metastore for stateful needs like deduplication and windowing. Stream processing pods combine ingestion triggers, a worker pool runtime, and user-provided domain logic plugins with customizable failure handling. This infrastructure scales to handle over 300 billion events weekly while maintaining workload isolation and elastic autoscaling.


### [Marionette - Enabling E2E User-scenario Simulation](https://yomu.fyi/post/marionette-enabling-e2e-user-scenario-simulation.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Anish Jha
- Published: Dec 23, 2019
- Type: Problem & solution

Conducting end-to-end testing across Grab's transport microservices became difficult due to service availability, environment construction, cross-service authentication, and complex data setups for real-world user accounts. To address these challenges without relying on physical mobile devices or emulators, Grab built Marionette, an internal simulation platform for passenger and driver interactions. The platform provisions required test data, coordinates booking lifecycles, and isolates test executions across distinct user groups using localized cohorts. Engineers can configure driver and passenger behaviors, execute workflows, and run load or integration tests through a dedicated user interface, a Go SDK, and RESTful APIs.


### [How We Implemented Domain-Driven Development in Golang](https://yomu.fyi/post/how-we-implemented-domain-driven-development-in-golang.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Kapil Chaurasia
- Published: Nov 21, 2019
- Type: Problem & solution

Building GrabPlatform's partner integration self-service portal initially resulted in an unstructured codebase where individual files exceeded 500 lines and lacked proper segregation. Modifying existing functions carried high risks of breaking functionality across imported source collections. To resolve this, the team restructured the Go application using Domain-Driven Design principles in coordination with product domain experts. They mapped business rules into bounded contexts, identified entities and aggregate roots, introduced repository interfaces, and utilized domain events for cross-context communication. The refactoring distributed core functionality evenly, simplified onboarding, and aligned technical terminology with business concepts.


### [Automated Continuous Deployment at Heroku](https://yomu.fyi/post/automated-continuous-deployment-at-heroku.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Bernerd Schaefer
- Published: Oct 22, 2019
- Type: Problem & solution

Heroku’s Runtime team replaced a manual, coordination-heavy deployment process after team growth and Common Runtime sharding expanded work from one infrequently changed application in two environments to multiple services across 12 environments. Existing Heroku Pipelines deploy merged, CI-passing changes to staging, while the Go-based cedar-service-deployer reconciles differences between pipeline stages and promotes them when a series of safety checks pass. Those checks include a 30-minute age gate, deployment and incident controls, master-branch verification, Librato alerts, and HTTP 200 checks for configured URLs, including platform integration tests. The stateless system can handle concurrent rollouts, and the team added practices such as isolated early stages, feature flags, and separated database-migration changes. After about two years, it managed 23 services, allowing engineers to merge and rely on healthy systems for global release.


### [Preventing Pipeline Calls from Crashing Redis Clusters](https://yomu.fyi/post/preventing-pipeline-calls-from-crashing-redis-clusters.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Michael Cartmell
- Published: May 5, 2019
- Type: Incident / postmortem

A single Redis slave node failure caused Grab's Apollo booking service to suffer an over 95 percent call failure rate for one minute despite running a three-shard cluster with two replicas per partition. Investigation revealed that the service configured Go-Redis to route all read queries exclusively to slave nodes to offload master CPU usage. When the slave node dropped offline, batched HMGET pipeline calls failed completely because the client wrapper treated a single command failure as a failure of the entire pipeline. Furthermore, the Go-Redis client cached cluster topology and only lazily refreshed state every sixty seconds, continuing to direct traffic to the dead replica until the timer expired. Grab addressed this risk by recommending dedicated pipeline clients configured with latency-based routing to allow reads to fall back to responsive master nodes.


### [Loki, a Dynamic Mock Server for HTTP/TCP Testing](https://yomu.fyi/post/loki-a-dynamic-mock-server-for-http-tcp-testing.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Thuy Nguyen
- Published: Apr 10, 2019
- Type: Problem & solution

Grab built Loki, a dynamic mock server written in Golang that simulates backend services on local developer machines and CI pipelines. Mobile app testing previously suffered from heavy dependencies on complex, brittle staging environments and interconnected services communicating over HTTP, HTTPS, and TCP. Loki handles both HTTP and TCP traffic on distinct ports while exposing a unified RESTful API to manage test expectations. It provides runtime flexibility through sandboxed JavaScript execution, configurable request sequence ordering, and an in-memory cron scheduler for TCP push messages. Adopting Loki decoupled mobile releases from staging stability, improving delivery cycles and enabling automated UI testing with Espresso and XCUITest.


### [Meet Shopify Developer Kirsten Westeinde  - Shopify](https://yomu.fyi/post/meet-shopify-developer-kirsten-westeinde-shopify.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 22, 2019
- Type: Explainer

Kirsten Westeinde’s just-over-five-year Shopify career spans internal tools, partner-facing applications, and backend development on Kit as the company scaled. She began with Ruby on Rails, later helped extract partner software from Shopify Core using Elasticsearch, Kafka, Ruby on Rails, and the Shopify API, and now works with Data Engineering on real-time recommendations. The feature combines Go and Ruby and is designed to handle the current load while scaling to future demand. Her problem-solving practices include whiteboarding system interactions, taking mental breaks, and writing or explaining ideas to align collaborators. She also describes mentorship, pair programming, a women’s developer meetup, and f(empower) as ways to support growth and address exclusion, bias, and microaggressions in technology.


### [Context Deadlines and How to Set Them](https://yomu.fyi/post/context-deadlines-and-how-to-set-them.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Michael Cartmell
- Published: Mar 11, 2019
- Type: Explainer

Microservice architectures operating under heavy network traffic require robust timeout handling to prevent slow or failing dependencies from causing cascading failures across services. Naive, static timeout configurations across a call chain often cause downstream components to waste compute effort on requests that upstream callers have already abandoned. To establish predictable timeout thresholds, engineers can align limits with service-level latency percentiles, pairing P99 limits with median latency retry allowances. Go's context package improves upon static network timeouts by propagating request-scoped deadlines and cancellation signals across service boundaries. This distributed context ensures downstream servers recognize remaining time budgets and terminate unneeded processing immediately when parent deadlines expire or callers manually cancel requests.


### [Structured Logging: The Best Friend You’ll Want When Things Go Wrong](https://yomu.fyi/post/structured-logging-the-best-friend-you-ll-want-when-things-go-wrong.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Aditya Praharaj
- Published: Mar 5, 2019
- Type: Problem & solution

Grab redesigned its backend logging approach to address mounting vendor costs, query language limitations, and debugging difficulties across a growing microservices ecosystem. Most services previously emitted syslog-style key-value logs almost entirely at the INFO level, which made volume reduction difficult and lacked causal ordering and automated correlation. The engineering team migrated to a self-managed Elastic stack backend and built a structured logging library in Go from the ground up. This framework introduces dynamic log-level adjustment at runtime, automatic trace-based log correlation via Grab-Kit, and the Common Grab Log Schema to enforce consistent JSON formatting without Elasticsearch indexing conflicts.


### [How We Simplified Our Data Ingestion & Transformation Process](https://yomu.fyi/post/how-we-simplified-our-data-ingestion-transformation-process.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Yichao Wang
- Published: Mar 3, 2019
- Type: Problem & solution

Grab evolved its real-time data ingestion pipeline after an initial architecture built on Spark Streaming and Python encountered operational complexity, node failures, and data loss from S3 eventual consistency. Because the streaming workload primarily handled event partitioning and ORC file generation, the team consolidated these tasks directly into an existing Golang processing service. They implemented sharded concurrent maps for high-throughput partitioning and optimized heap allocations to resolve memory bottlenecks. This refactor removed intermediate Avro conversions and intermediate storage hops. The simplified Go pipeline eliminated data loss and reduced processing lag from up to 13 minutes down to approximately 1 minute.


### [Querying Big Data in Real-time with Presto & Grab's TalariaDB](https://yomu.fyi/post/querying-big-data-in-real-time-with-presto-grab-s-talariadb.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Roman Atachiants
- Published: Jan 2, 2019
- Type: Problem & solution

Grab developed TalariaDB to support real-time SQL querying over high-velocity event streams while maintaining predictable sub-second latencies and low infrastructure costs. The distributed time-series store retains only the most recent hour of data and integrates directly with Presto via its PrestoThriftService interface. Internally, TalariaDB uses the Go-based Badger key-value store to maintain an in-memory key index of metric names and timestamps while mapping columnar event payloads directly to disk. Ingestion occurs by processing pre-partitioned event batches written to Amazon S3 via SQS notifications. By combining a zero-copy decoder with parallel split evaluation across gossiping cluster nodes, the architecture scales horizontally while serving millions of events per second.


### [Designing Resilient Systems: Circuit Breakers or Retries? (Part 1)](https://yomu.fyi/post/designing-resilient-systems-circuit-breakers-or-retries-part-1.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Corey Scott
- Published: Dec 21, 2018
- Type: Explainer

Distributed architectures frequently encounter upstream failures triggered by networking issues, system overloads, resource starvation, and invalid deployments. Implementing software circuit breakers interposes a monitoring mechanism between components to halt requests when failure thresholds are met, giving struggling upstream dependencies time to recover. Circuit breakers save CPU, memory, and network resources by failing fast or routing execution through defined fallbacks such as cached data, alternate services, or approximation algorithms. Grab utilizes Hystrix-Go to manage upstream interactions and configure key thresholds for concurrency, timeouts, and error ratios. This approach protects downstream consumers from cascading latency while insulating upstream resources from excess traffic.


### [Reliable and Scalable Feature Toggles and A/B Testing SDK at Grab](https://yomu.fyi/post/reliable-and-scalable-feature-toggles-and-a-b-testing-sdk-at-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Roman Atachiants
- Published: Nov 2, 2018
- Type: Problem & solution

Grab previously managed experiments using custom service-level code and a toggling library that queried a shared Redis instance, creating latency risks and a single point of failure across backend microservices. To achieve reliable, sub-microsecond feature evaluations, the team designed a Go SDK that resolves rollouts and A/B tests entirely in memory without runtime network I/O. Backend services periodically poll JSON-defined configuration schemas stored in Amazon S3 through a Universal Configuration Manager. The SDK evaluates contextual attributes called facets locally and pushes decision telemetry asynchronously to an S3 and Presto data lake. This architecture allows engineering and product teams to gate deployments and run server-side experiments safely without service disruption.


### [Mockers - Overcoming Testing Challenges at Grab](https://yomu.fyi/post/mockers-overcoming-testing-challenges-at-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Mayank Gupta
- Published: Sep 18, 2018
- Type: Problem & solution

Grab operates over 250 microservices communicating over HTTP and gRPC, making shared staging environments costly, ambiguous in ownership, and fragile due to inconsistent data and uncoordinated deployments. To address these testing bottlenecks, Grab created Mockers, a Go SDK and CLI tool backed by a central monorepo of mock servers for local-box and CI testing. Mockers automatically generates HTTP and gRPC mock servers from Swagger specifications and protobuf files, returning configured network responses without internal business logic. By incorporating Grab's in-house chaos SDK middleware, Mockers also enables repeatable resiliency and contract testing locally without relying on code-level mocks. While Grab still mandates integration testing on distributed staging environments with live data, Mockers enables developers to detect complex defects and contract mismatches earlier.


### [Applying Seccomp Filters at Runtime for Go Binaries](https://yomu.fyi/post/applying-seccomp-filters-at-runtime-for-go-binaries.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: chris le roy
- Published: Aug 29, 2018
- Type: Tutorial

Seccomp filters let a Linux process restrict its available system calls, reducing the process attack surface beyond the Dyno’s existing restrictions. This tutorial builds a Go binary that creates /tmp/moo, uses strace -c to identify its runtime syscalls, and configures a whitelist with libseccomp-golang. The filter starts with libseccomp.ActErrno returning EPERM for unlisted calls, then adds ActAllow rules for named calls before loading the filter at runtime; the example also checks kernel support and installs libseccomp-dev. With the listed calls enabled, the program creates the folder successfully, while a later syscall.Exec attempt to run ls -l cannot perform unlisted ioctl, getdents, and statfs operations and reports “Operation not permitted.” The post then describes deploying the dependency with Govendor or Godeps on Heroku.


### [Building Grab’s Experimentation Platform](https://yomu.fyi/post/building-grab-s-experimentation-platform.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Abeesh Thomas
- Published: Jul 13, 2018
- Type: Problem & solution

Grab built its internal Experimentation Platform (ExP) to replace a manual, expensive testing process that required bespoke meetings, custom logging pipelines, and service modifications for each experiment. ExP provides a unified infrastructure featuring a centralized management UI, automated real-time data streaming to S3, and SDKs for Android, iOS, and Go. The platform leverages JSON-based experiment definitions delivered through dynamic configuration management, enabling client-side evaluation without costly network calls. It addresses marketplace network effects and inter-experiment interference through mechanisms such as geo-temporal segmentation and domain-layer models. The platform has scaled to run approximately 25 concurrent experiments while computing roughly 2,500 metrics and 50,000 experiment-metric combinations daily.


### [Introducing Grab-Kit: Distributed Service Design at Grab](https://yomu.fyi/post/introducing-grab-kit-distributed-service-design-at-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Karen Kue
- Published: Jun 8, 2018
- Type: Problem & solution

As Grab migrated from a monolith to microservices, maintaining consistency, coordination, and code quality across rapidly expanding teams became a major engineering challenge. To address this, the Developer Experience team built Grab-Kit, a Go framework that automates service scaffolding, code generation, and distributed system design patterns. The framework uses Protocol Buffer definition files as a single source of truth to generate data transfer objects, communication bindings, and standardized middleware for logging and profiling. Grab-Kit also features declarative metrics definitions that synchronize with the DataDog API to build and update service dashboards automatically. Adopting the framework reduced development time for creating new services by up to 70% in teams such as GrabFood while improving overall system stability.


### [Scaling ipify to 30 Billion Requests and Beyond on Heroku](https://yomu.fyi/post/scaling-ipify-to-30-billion-requests-and-beyond-on-heroku.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Randall Degges
- Published: Jan 23, 2018
- Type: Problem & solution

ipify began as a free API that returns a caller’s public-facing IP address, created to help discover cloud instances’ external addresses without management APIs. Its initial implementation was a sub-50-Loc Node service deployed on one Heroku dyno for about $7/month; rising usage caused slow responses and 503s. Investigation showed roughly 10 rps, while one Node process and string manipulation around X-Forwarded-For limited throughput; clustering doubled capacity, and the implementation later used Go. Heroku supplied load balancing, automatic restarts, and horizontal dyno scaling as traffic expanded through bursty demand and malicious use; the operator chose manual capacity rather than paid autoscaling. The service surpassed 30B requests per month, routinely serving 2k–20k RPS with 1–20ms average response times at $150–$200 monthly; ownership later passed to whoisxmlapi.com, whose work includes an improved UI and more data endpoint.


[Newer posts](https://yomu.fyi/topic/go/page/3.md) · [Older posts](https://yomu.fyi/topic/go/page/5.md)