# Streamlining Grab's Segmentation Platform with faster creation and lower latency

[Grab](https://yomu.fyi/company/grab) · Jake Ng · Aug 15, 2023

**Type:** Problem & solution

## Summary

Grab's Segmentation Platform previously stored user-to-segment mappings across individual rows in ScyllaDB, causing write bottlenecks during segment creation and read latencies too high for downstream consumers. To resolve these performance limitations, the team transitioned to storing segments as Roaring Bitmaps saved as single blobs in object storage. This compression strategy splits 32-bit integer user IDs into chunks across array, bitmap, and run containers based on data density, reducing a one-million-member segment to under one megabyte. An accompanying client SDK manages segment retrieval, decoding, update notifications, and least-recently-used in-memory caching. Consequently, consumers such as Grab's communications platform achieved peak throughput of 15,000 queries per second with sub-millisecond p99 read latencies.

## Context

Grab's Segmentation Platform stored user segment mappings across individual rows in ScyllaDB, which caused write bottlenecks that delayed segment creation by hours and produced read latencies too slow for latency-critical services like communications and experimentation.

## Approach / What changed

Grab transitioned segment storage to Roaring Bitmaps stored as single blobs in object storage and provided a client SDK that handles local retrieval, decoding, update watching, and LRU memory caching.

## Takeaways

- Roaring Bitmaps partition 32-bit integers by using the upper 16 bits as chunk indexes and encoding the lower 16 bits into array, bitmap, or run containers based on data distribution.
- Storing segments as bitmaps enables downstream services to perform dynamic set operations such as unions, intersections, and differences on the fly without database re-materialization.
- Using an SDK with local LRU caching and automated refresh subscriptions enabled Grab's communications platform to achieve 15,000 QPS with sub-millisecond p99 latency.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Caching](https://yomu.fyi/topic/caching), [Go](https://yomu.fyi/topic/go), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://engineering.grab.com/streamlining-grabs-segmentation-platform)