Using Betas to Deploy New Features Safely

Summary

Continuous deployment makes changing a running system risky, especially for a platform serving more than a million merchants and changing code multiple times daily. The post presents beta flags as a control mechanism: per-Subject BetaFlags support explicit activation, while BetaRollouts use a percentage and a digest-based modulo calculation to assign a stable, growing subset as rollout increases. It then places a higher-level Feature abstraction above those primitives, allowing direct targeting, percentage rollout, per-Subject opt-outs, and a 100% kill switch that can halt a feature without deleting thousands of database records. The guidance also covers testing permutations, notifying owners through BetaIdentifier metadata, and removing stale flags after rollout. These patterns provide operational control and safer deployment, while requiring substantial implementation and ongoing cleanup.

Takeaways

1. 1

   A digest of the Subject and beta identifiers, reduced modulo 100, keeps rollout membership stable as the percentage increases, so previously enabled Subjects remain enabled.

2. 2

   A percentage rollback can set BetaRollout to 0, but explicit BetaFlags remain active; a higher-level opt-out rollout at 100% provides a kill switch when direct records are difficult to remove.

3. 3

   Feature flags add code-path permutations. Teams may run selected test files twice for complex features, audit tests when flags are removed, and clean up stale flags to avoid dead code and technical debt.