Loading…
StreamBuilder: our open-source framework for powering your dashboard.
engineering
Summary
Tumblr has open-sourced StreamBuilder, a custom framework developed and utilized for years to power content feeds across its platform. The framework underlies user experiences such as Following feeds, For You feeds, search results, tagged post lists, and blog recommendations. StreamBuilder's architecture organizes content into distinct streams, allowing engineers to mix, filter, inject recommendations at intervals, and rank items for relevancy or engagement. The released package includes a YAML composition syntax, abstractions for debugging, stream-within-stream compositions like carousels, cursor-based pagination, and unit tests covering the public interface. Future additions planned for the open-source repository include migrated documentation, further stream template examples, and common stream implementations.
Context
Tumblr feeds—including Following, For You, search results, tagged posts, and blog recommendations—require combining separate streams of content with distinct logic, interval injections, block filtering, and relevancy ranking under a unified framework.
Approach / What changed
Tumblr open-sourced StreamBuilder on GitHub, releasing its core library alongside a YAML syntax for stream composition, abstractions for filtering, ranking, debugging, cursor-based pagination for complex stream templates, support for stream-within-stream carousels, and public interface unit tests.
Takeaways
- StreamBuilder models feeds as distinct content streams that can be mixed, filtered by criteria like blocked users, and ranked for relevancy or engagement.
- The open-sourced package provides a YAML syntax to compose streams and define how content is filtered, injected, and ranked.
- The library includes built-in abstractions for cursor-based pagination across complex stream templates and composite stream structures such as carousels.
Related reading
engineering.tumblr.com ·
How Reblogs Work
Tumblr reblog trees expand from an original root post into branching reblog trails as users reshare and comment on content. Storing these trails traditionally using relational database joins would require numerous queries per post on trails that extend thousands of levels deep. To maintain a constant query load of one query per post, Tumblr copies reblog trail content on every reblog rather than fetching intermediate posts at runtime. Older posts stored this copied content within deeply nested HTML blockquotes relying solely on permalink URLs, which caused trails to break whenever blogs changed their names. With the Neue Post Format, Tumblr moved to a JSON array structure that embeds trail content alongside immutable references to each blog and post, reducing broken trails while preserving single-query post loading.
engineeringengineering.tumblr.com ·
Tumblr Hack Day, March 2023 Edition
Tumblr hosts internal Hack Day events several times each year, temporarily pausing regular product work to let engineers build experimental features. During the March 2023 session, one project integrated LibreTranslate to provide post text translation across the platform for global users. Another hack introduced an Android Feature Wishlist prototype, allowing users to view and upvote separate lists of staff and community feature proposals. Additional prototypes clarified reblog tag attribution by displaying the reblogger avatar next to added tags, and modernized 1:1 messaging using alternating chat bubbles with updated colors. These experimental projects demonstrate potential site updates that Tumblr may test and eventually launch into production.