---
title: "Engineering stories behind the Medium Daily Digest Algorithm: Part 2"
description: "Medium’s Daily Digest filtering had become costly and difficult to maintain after a single Bloom Filter instance accumulated billions of items for read, muted, and presentation rules. The system could face up to 15,000 filter questions per feed and more than 15 million per second across thousands of feeds, while Bloom Filters could not delete or enumerate entries and introduced derived-data drift. The team replaced the muted-writer filter with a user-based database query that fetched all muted writers for the current user and cross-referenced them with the story shortlist, then split read filtering between early user-based queries and later pairwise lookups. The resulting implementation removed Bloom Filters entirely, was reported as ten times cheaper in their situation, made costs independent of shortlist size, supported reading-history clearing, and was easier to control and debug."
---

# Engineering stories behind the Medium Daily Digest Algorithm: Part 2

[Medium](https://yomu.fyi/company/medium) · Raphael Montaud · Aug 28, 2025

**Type:** Problem & solution

## Summary

Medium’s Daily Digest filtering had become costly and difficult to maintain after a single Bloom Filter instance accumulated billions of items for read, muted, and presentation rules. The system could face up to 15,000 filter questions per feed and more than 15 million per second across thousands of feeds, while Bloom Filters could not delete or enumerate entries and introduced derived-data drift. The team replaced the muted-writer filter with a user-based database query that fetched all muted writers for the current user and cross-referenced them with the story shortlist, then split read filtering between early user-based queries and later pairwise lookups. The resulting implementation removed Bloom Filters entirely, was reported as ten times cheaper in their situation, made costs independent of shortlist size, supported reading-history clearing, and was easier to control and debug.

## Context

Medium’s recommendation filtering used a single, ever-growing Bloom Filter instance for read, muted, and presentation rules. The filter accumulated billions of items, could not delete or enumerate entries, made changes such as unmuting difficult, and required derived data to be synchronized with the source database. Filtering also involved up to 15,000 questions for one feed and more than 15 million questions per second across thousands of feeds.

## Approach / What changed

The team replaced Bloom Filter-based filtering with user-based database queries. For muted writers, it fetched all muted writers for the current user and cross-referenced them with the story shortlist. It split read filtering into early user-based queries and later pairwise user-post lookups, using each at a different stage of recommendation generation.

## Takeaways

- Bloom Filters supported fast membership checks but could not list or delete entries, so an unmuted writer could remain effectively muted and stored data could drift from the source database.
- For most users, querying all muted writers from DynamoDB retrieved fewer than one item per feed on average; users who muted thousands of writers could still be handled in real time, with higher latency.
- The new implementation removed Bloom Filters entirely and was reported as ten times cheaper in this situation, while making filtering costs independent of the initial shortlist size and enabling reading-history clearing.

**Tags:** [Bloom Filters](https://yomu.fyi/topic/bloom-filters), [Databases](https://yomu.fyi/topic/databases), [DynamoDB](https://yomu.fyi/topic/dynamodb), [Recommendation Systems](https://yomu.fyi/topic/recommendation-systems)

- Source: [Medium](https://medium.engineering/engineering-stories-behind-the-medium-daily-digest-algorithm-part-2-c977ad0b134f)
- Source URL: https://medium.engineering/engineering-stories-behind-the-medium-daily-digest-algorithm-part-2-c977ad0b134f
- Ingested by Yomu: 2026-09-09T16:02:49.761Z

[Read original post](https://medium.engineering/engineering-stories-behind-the-medium-daily-digest-algorithm-part-2-c977ad0b134f)