# Scaling Nextdoor’s Datastores: Part 5

[Nextdoor](https://yomu.fyi/company/nextdoor) · Slava Markeyev · Mar 19, 2025

**Type:** Problem & solution

## Summary

Nextdoor addressed database and cache consistency issues caused by missed cache writes and concurrent read-fill operations in their look-aside architecture. While forward row versioning prevents out-of-order write inconsistencies, writer failures and race conditions during cache misses can leave stale data persisted in Redis. To resolve this, Nextdoor built a reconciliation pipeline that consumes PostgreSQL WAL replication logs with pg-bifrost, streams changes through Apache Kafka, and executes conditional deletions in Redis. The Go-based reconciler operates in two passes using a time wheel, running one pass in near real time and a second pass after a delay exceeding web request timeouts. Because conditional deletion evaluates row versions directly in the cache, the system processes change streams out of order and scales horizontally.

## Context

Forward cache versioning handles racing writes but fails if writers cannot communicate with the cache or if a reader fills stale data during an in-flight write. When a write to the database succeeds but fails to update the cache, the cache becomes inconsistent unless an external process intervenes. Additionally, concurrent reads during missed writes can overwrite fresh invalidations with stale database snapshots.

## Approach / What changed

Nextdoor built an asynchronous reconciliation pipeline consuming PostgreSQL WAL logs via pg-bifrost and Apache Kafka. A Go-based reconciler processes the change stream and executes conditional cache deletions (del\_if\_version) in Redis based on row versions. To address cache-fill race conditions, the system uses a two-pass reconciliation approach implemented with a time wheel: one pass applies invalidations in near real time, while a second pass runs with a fixed delay slightly higher than the web request timeout.

## Takeaways

- Using database row versions for conditional cache deletions (del\_if\_version) removes the requirement for strictly ordered change stream processing, enabling horizontal scaling of reconciler workers.
- A two-pass reconciliation mechanism mitigates race conditions during cache miss repopulation by executing one invalidation pass immediately and a second pass delayed beyond the web request timeout.
- The reconciliation pipeline combines pg-bifrost to capture PostgreSQL WAL logs, Apache Kafka as a persistent message bus, and a Go-based consumer issuing conditional Redis deletions.

**Tags:** [Caching](https://yomu.fyi/topic/caching), [Kafka](https://yomu.fyi/topic/kafka), [Postgres](https://yomu.fyi/topic/postgres), [Redis](https://yomu.fyi/topic/redis), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://engblog.nextdoor.com/scaling-nextdoors-datastores-part-5-5221da60f374)