How Uken Games reduces observability costs by 87% with ClickHouse

Summary

Uken Games replaced Datadog with an open-source observability stack built around ClickHouse to monitor backend services supporting mobile games with millions of players. The migration addressed rising Datadog costs and vendor lock-in while targeting pragmatic feature parity across metrics, traces, and logs. OpenTelemetry collectors run in two layers: lightweight agents on ECS instances pre-filter traces, while an autoscaling gateway batches and buffers writes to SigNoz on ClickHouse; managed Prometheus handles metrics, CloudWatch retains logs, and Grafana provides dashboards and alerts. Uken samples ordinary traces, retains errors and high-latency requests, computes alert metrics without sampling, and limits retention to two weeks. The resulting system stores all traces on one ClickHouse node using about 170 GB, operates without migration downtime, and costs about $12,000 annually—an 87% reduction.

Takeaways

1. 1

   A single ClickHouse node stores all of Uken’s traces using about 170 GB of disk, while Grafana runs roughly 600 queries per minute across ClickHouse, CloudWatch, and Prometheus.

2. 2

   Uken keeps errors and high-latency traces, samples other traces at a representative rate such as 5–10%, and computes alerting metrics without sampling to avoid losing operational signals.

3. 3

   The new open-source observability system costs about $12,000 per year, reduces observability costs by 87%, and was rolled out with zero downtime.