# Round-robin in Distributed Systems

[Grab](https://yomu.fyi/company/grab) · Gao Chao · Sep 27, 2016

**Type:** Explainer

## Summary

Building client-side load balancing for Grab's Common Data Service prompted a move from AWS Elastic Load Balancers to DNS discovery due to persistent connection issues and unpredictable scaling events. After patching an open-source library that failed to rotate IP sequences properly, the author evaluated different Go patterns for round-robin routing. A mutex-protected array counter provides the simplest model for basic retrieval, though adding mutations requires careful lock coordination. Alternatively, a dedicated balancer goroutine receiving requests over nested channels enables explicit operation timeouts and centralized event handling at the cost of higher code complexity and channel creation overhead. The author recommends the mutex approach for resource fetching and the goroutine-based design for workload balancing.

## Context

Grab's Common Data Service needed client-side load balancing between clients and servers, but persistent connection issues, suboptimal performance, and unpredictable scaling events with AWS Elastic Load Balancers led to adopting DNS discovery.

## Approach / What changed

Evaluating and implementing round-robin algorithms in Go using two patterns: a sync.Mutex protecting an array index, and a goroutine multiplexing operations over Go channels with timeout handling.

## Takeaways

- A mutex-based round-robin implementation is simple and fast for single-action structs, but extra care is required when adding update or set operations.
- A channel- and goroutine-based round-robin structure enables granular operation timeouts and centralized state management across multiple event cases.
- Channel-based balancing introduces nanosecond-scale latency overhead per operation because channels are created for each request.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [AWS](https://yomu.fyi/topic/aws), [Go](https://yomu.fyi/topic/go), [Reliability](https://yomu.fyi/topic/reliability), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://engineering.grab.com/round-robin-in-distributed-systems)