# Scaling Like a Boss with Presto

[Grab](https://yomu.fyi/company/grab) · Aneesh Chandra · May 1, 2017

**Type:** Problem & solution

## Summary

Grab experienced severe performance degradation, long queue times, and connection timeouts on its Amazon Redshift analytics cluster as user concurrency and reporting workloads expanded. Although an initial Amazon S3 data lake decoupled storage from compute, business users required standard SQL interfaces rather than Spark data pipelines. The team deployed Presto clusters on AWS EMR, switching their storage format from AVRO to Parquet to support ANSI SQL querying directly against S3. Utilizing a shared Hive metastore on Amazon RDS allowed Grab to adopt a shared-data multi-cluster architecture that isolated distinct workloads across dedicated compute clusters. This setup enabled rapid cluster scaling, streamlined failover, and matched Redshift performance on partitioned time-range queries.

## Context

Grab's Amazon Redshift analytics cluster struggled to handle growing query workloads, report concurrency, and peak demand periods, leading to long wait times, connection timeouts, and operational bottlenecks.

## Approach / What changed

Grab implemented an S3-based data lake with Parquet storage and deployed isolated Presto clusters on AWS EMR connected to a unified Hive metastore on Amazon RDS.

## Takeaways

- Adopting a shared-data multi-cluster Presto architecture over S3 enables workload isolation, rapid horizontal scaling, and standby cluster failovers without replicating data.
- Switching storage from AVRO to Parquet with granular time-based S3 partitions allowed Presto to achieve query speeds comparable to Redshift for time-bounded analytical workloads.
- Updating mutable datasets within the S3 data lake required a recursive data processing (RDP) pattern using Spark jobs orchestrated by an in-house dependency scheduler.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [AWS](https://yomu.fyi/topic/aws), [Data Pipelines](https://yomu.fyi/topic/data-pipelines), [Migrations](https://yomu.fyi/topic/migration), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://engineering.grab.com/scaling-like-a-boss-with-presto)