# Alpha Launch Postmortem

[Supabase](https://yomu.fyi/company/supabase) · Paul Copplestone · Jul 10, 2020

**Type:** Incident / postmortem

## Summary

An unexpected viral submission on Hacker News prompted sudden alpha traffic for Supabase, attracting 30,000 visitors and over 1,400 signups in one week. The surge strained initial infrastructure, which initially relied on a single server running Docker Compose for middleware and Digital Ocean for customer databases. System bottlenecks emerged rapidly, including hit limits on Digital Ocean droplet counts, frequent physical server faults, and a three-hour provisioning outage when Cloudflare reached its 1,000-subdomain limit. In response, the team migrated middleware to multiple globally load-balanced ECS clusters via AWS Global Accelerator. Supabase also transitioned 1,800 servers from Digital Ocean to AWS t3a instances, nearly halving database startup times from 90 seconds to 50 seconds.

## Context

During alpha testing, an unplanned viral Hacker News post drove 30,000 visitors and over 1,400 signups in seven days, pushing Supabase to over 1,000 customer databases. This unexpected traffic surge exposed infrastructure bottlenecks across third-party providers, including rapid exhaustion of Digital Ocean server limits, physical hardware errors requiring offline migrations, and a three-hour outage caused by hitting Cloudflare's 1,000-subdomain threshold.

## Approach / What changed

Supabase migrated its middleware from a single Ubuntu server running Docker Compose to multiple ECS clusters load-balanced with AWS Global Accelerator. To address cloud provider issues and credit limitations on Digital Ocean, the team migrated 1,800 servers to AWS under the AWS Activate program, deploying on AWS t3a instances to improve Postgres provisioning speed.

## Takeaways

- Hitting Cloudflare's 1,000-subdomain limit caused a three-hour outage because account upgrade permissions were restricted solely to the account owner, preventing available team members from resolving it immediately.
- Running middleware via Docker Compose on a single 4-CPU, 8GB RAM Ubuntu server unexpectedly handled the initial viral traffic before being transitioned to multiple ECS clusters using AWS Global Accelerator.
- Migrating 1,800 database servers from Digital Ocean to AWS t3a instances reduced Postgres database startup times from approximately 90 seconds to 50 seconds.

**Tags:** [AWS](https://yomu.fyi/topic/aws), [Docker](https://yomu.fyi/topic/docker), [Incident Response](https://yomu.fyi/topic/incident-response), [Migrations](https://yomu.fyi/topic/migration), [Postgres](https://yomu.fyi/topic/postgres)

[Read original post](https://supabase.com/blog/alpha-launch-postmortem)