Shopify2023-10-18Problem & solution
Automatically Rotating GitHub Tokens (So You Don’t Have To)
Summary
Shopify needed automated GitHub workflows that could clone from private repositories, update organization-level projects, and trigger downstream workflows but could not use GITHUB_TOKEN. Personal access tokens supplied those capabilities, yet introduced developer churn, user-attributed actions, and no organization-enforced rotation or expiry. The proposed system uses GitHub Actions and a GitHub App to generate repository secrets that rotate automatically while preserving PAT-like references in existing workflows. Its serverless design delegates scheduling, managed infrastructure, approvals, secret management, auditing, and break-glass operation to GitHub, while a custom TypeScript action encrypts and writes generated secrets. Prototyping exposed that parallel one-second jobs could accrue multiple billable minutes, prompting sequential execution in a consolidated workflow; the stated goal is improved resilience to developer churn and reduced leaked-secret blast radius.
Context
Shopify’s automated workflows needed to access repo-external resources, push issues or pull requests into organization-level projects, and trigger downstream workflows, which precluded using GITHUB_TOKEN. Personal access tokens addressed those use cases but were vulnerable to developer churn, attributed actions to individual users, and lacked organization-enforced rotation or expiry requirements.
Approach / What changed
The system uses scheduled and manually triggerable GitHub Actions, a GitHub App for generating secrets, repository secrets for PAT-compatible consumption, and a custom TypeScript action that creates installation tokens, retrieves the repository public key, encrypts secrets, and writes them to the target repository. Customer workflows are consolidated and run sequentially to control Action costs, with optional onboarding, canary testing, and operational metrics.
Takeaways
GitHub Apps provide separate credentials, names, and secrets, making automations more resilient to developer churn and avoiding attribution to personal developer accounts.
GitHub Actions billing rounds execution duration at the job level, so parallel jobs can accumulate multiple billable minutes even when each runs for about one second.
Generated secrets are placed in repository secrets, allowing existing workflows to reference them like PATs while rotations occur transparently in the background.
Source and freshness
- Source
- shopify.engineering
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.