Loading post…
GithubTaylor CuiltyProblem & solution
From 2,500 requests to 24: Rebuilding star history without the stargazers
Summary
The post explains GitHub’s redesign of the stargazers API to provide star history without exposing user identities. The original endpoint returned paginated lists of stargazers, which tools used to reconstruct growth curves by fetching all pages, leading to privacy risks and expensive database queries that counted all stars on each request due to a pagination bug. GitHub first restricted access to stargazer lists to repository collaborators, but this broke tools that needed star history. Collaborating with maintainers and the Star History project, GitHub built a new endpoint that returns up to 30 weeks of daily star counts in a single request with no identities attached. The new endpoint reduces database work by filtering stars by time before aggregation, cutting requests for repositories like react/react from about 2,511 to 24.
Context
GitHub’s original stargazers endpoint returned paginated lists of users who starred a repository, along with timestamps when requested via a custom media type. Tools used this to reconstruct star history, but the endpoint was designed for listing people, not time-series aggregation. This created two problems: bulk harvesting of identities for spam, and expensive database queries that counted all stars per request due to a pagination bug. After restricting access to collaborators, GitHub worked with tool maintainers to create a purpose-built endpoint for star history.
Approach / What changed
A new endpoint GET /repos/{owner}/{repo}/star_history returns up to 30 weeks of daily star counts in a single request. It paginates by calendar week: each request translates page and per_page into an indexed created_at range, filters stars to those weeks, then groups matching records into daily counts. This avoids scanning and aggregating all current stars, reducing requests for large repositories from thousands to about 24.
Takeaways
A bug in the original stargazers endpoint caused every request to count all current stars for the repository, even when returning only one page, leading to slow queries that averaged nearly 3 seconds and frequent timeouts.
The new endpoint reduces database load by filtering stars by an indexed time range before grouping and counting, rather than aggregating all stars and then paginating.
The same time-range filter approach was applied to the repository stargazers page on GitHub.com and other stargazer API endpoints, eliminating slow count queries.
FAQ
Why did the original stargazers endpoint become expensive for tools that needed star history?
The endpoint was designed to list people, not time-series. Each request counted all current stars due to a pagination bug, and tools had to fetch up to 2,511 pages for large repos like react/react.
How does the new star history endpoint avoid privacy risks?
It does not return any user identities. It only returns daily star counts for up to 30 weeks per request, with no usernames or other personal data.
What remains a limitation of the new endpoint?
The time-range design does not limit the number of stars within a single week, so an unusually dense week can still produce a slow or expensive query.
Source and freshness
- Source
- github.blog
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.