---
title: "Evolving the Backend Storage for Platform Metrics"
description: "Heroku migrated MetaaS, its internal Metrics as a Service platform metrics backend, from self-hosted Apache Cassandra to AWS’s DynamoDB. MetaaS collects application observations, aggregates them into per-application, per-minute statistics such as median, maximum, and 99th-percentile response time, and stores time-series data for dashboard metrics, alerting, and autoscaling. The migration first duplicated Kafka stream-processing writes into DynamoDB, then used “Science Mode” to compare Cassandra and DynamoDB read results in production without changing returned data. After edge cases were fixed and results matched, retention-period TTL expiry avoided a separate historical data transfer, while gradual query cutovers provided another safeguard. Since 100% of MetaaS queries moved to DynamoDB in May of last year, DynamoDB has remained reliable at scale and been faster and cheaper than the Cassandra clusters, with no noticeable effect on users."
---

# Evolving the Backend Storage for Platform Metrics

[Heroku](https://yomu.fyi/company/heroku) · David Murray · May 9, 2024

**Type:** Problem & solution

## Summary

Heroku migrated MetaaS, its internal Metrics as a Service platform metrics backend, from self-hosted Apache Cassandra to AWS’s DynamoDB. MetaaS collects application observations, aggregates them into per-application, per-minute statistics such as median, maximum, and 99th-percentile response time, and stores time-series data for dashboard metrics, alerting, and autoscaling. The migration first duplicated Kafka stream-processing writes into DynamoDB, then used “Science Mode” to compare Cassandra and DynamoDB read results in production without changing returned data. After edge cases were fixed and results matched, retention-period TTL expiry avoided a separate historical data transfer, while gradual query cutovers provided another safeguard. Since 100% of MetaaS queries moved to DynamoDB in May of last year, DynamoDB has remained reliable at scale and been faster and cheaper than the Cassandra clusters, with no noticeable effect on users.

## Context

MetaaS depended on home-grown Cassandra clusters whose routine maintenance had become increasingly difficult, and a particularly rough upgrade in a test environment showed that operational changes were needed. The system handled hundreds of thousands of observations per second, tens of thousands of writes per second, and thousands of read queries per second at peak.

## Approach / What changed

The migration used parallel stream-processing jobs to write the same time-series data to DynamoDB, followed by Science Mode dual reads that compared Cassandra and DynamoDB results without changing responses. After fixing edge cases, the team relied on TTL-based retention expiry instead of moving historical data, then gradually shifted production reads to DynamoDB.

## Takeaways

- MetaaS processes a couple hundred thousand observations per second and stores many terabytes of time-series data, with tens of thousands of writes and several thousand reads per second at peak.
- Science Mode queried DynamoDB in the background while Cassandra continued serving responses, exposing edge cases before the new backend became authoritative.
- TTL-based retention meant older Cassandra data could age out naturally, avoiding a lift-and-shift; DynamoDB was later reported to be faster, cheaper, and reliable at scale.

**Tags:** [DynamoDB](https://yomu.fyi/topic/dynamodb), [Kafka](https://yomu.fyi/topic/kafka)

- Source: [Heroku](https://www.heroku.com/blog/evolving-backend-storage-for-platform-metrics)
- Source URL: https://www.heroku.com/blog/evolving-backend-storage-for-platform-metrics
- Ingested by Yomu: 2026-09-10T00:01:58.584Z

[Read original post](https://www.heroku.com/blog/evolving-backend-storage-for-platform-metrics)