---
title: "Four Checkpoints for CDC: From Write-Ahead Log to Queryable Copy"
description: "Change data capture turns a database's write-ahead log into a continuously updated copy for analytics. This four-checkpoint walk-through shows how to wire it without stalling the primary."
---

# Four Checkpoints for CDC: From Write-Ahead Log to Queryable Copy

**Type:** [How to](https://yomu.fyi/visual-explainers/category/how-to.md)

![A four-step left-to-right diagram: a Postgres write-ahead log feeds a managed replication pipeline, then incremental materialized views, then a queryable analytics table, with checkpoint callouts on latency, load, and source protection.](https://yomufyi.s3.us-east-1.amazonaws.com/visual-explainers/bf7a48be-c05f-4906-85fb-63d18f4850dc/image-d56bf4cc-2599-484a-809e-722149fc706d.webp)

> Change data capture turns a database's write-ahead log into a continuously updated copy for analytics. This four-checkpoint walk-through shows how to wire it without stalling the primary.

## Explanation

Change data capture reads the log a database already writes and turns it into a continuously updated second copy. A practical path has four checkpoints: capture from the physical write-ahead log, move through a managed replication pipeline, reshape on arrival with incremental materialized views, and protect the source while reconciling schema and index mismatches. Done well it collapses an OLTP/OLAP split without adding a second write path: Fountain cut analytics latency from three hours to under two minutes while continuously replicating billions of CDC rows. The same mechanism also runs in reverse, syncing curated tables back into a serving database for low-latency reads.

## FAQ

### Should I capture CDC from the write-ahead log or by polling tables for changes?

Log-based capture reads what the database already writes, so it adds no query load to the source. In ClickHouse's Open House demo, WalShadow replicated Postgres into ClickHouse from the physical WAL with sub-second latency and stayed under 400 ms behind during a simulated market of 50,000 trades per second.

### Where does CDC latency actually come from?

It comes from the capture and delivery mechanism, not just the destination. In the same demo, a fraud burst was detected in under 500 ms through the physical-WAL engine but took over 15 seconds through the managed ClickPipes path, so measure your specific pipeline against your freshness target before committing to it.

### What breaks when you turn CDC on against a production Postgres?

Schema and index differences surface during replication: Shipsidekick hit a partial unique index conflict on its ClickPipes migration and resolved it by dropping and recreating the index. They also put timeouts on a read-only role so analytical and agent queries could not degrade primary performance.

**Tags:** [CDC](https://yomu.fyi/topic/cdc.md), [ClickPipes](https://yomu.fyi/topic/clickpipes.md), [OLTP](https://yomu.fyi/topic/oltp.md), [Postgres](https://yomu.fyi/topic/postgres.md), [Real-Time Analytics](https://yomu.fyi/topic/real-time-analytics.md), [Replication](https://yomu.fyi/topic/replication.md)

## Related reading

### [Real-time energy trading and analytics on Databricks](https://yomu.fyi/post/real-time-energy-trading-and-analytics-on-databricks.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Victoria Sicking, Jack Yallop
- Published: Oct 23, 2026
- Type: Problem & solution

Energy trading desks face increasing price volatility and the EU’s move from hourly to 15-minute market intervals, multiplying market periods by four. Separating operational and analytical systems creates two versions of truth, stale risk controls, duplicated governance, and extra synchronization. Databricks addresses this with a unified platform: Spark Structured Streaming Real-Time Mode streams market data with millisecond latency into Lakebase, a serverless Postgres serving as the operational database. The same governed data supports real-time trading apps in Databricks Apps, machine learning models for price forecasts, and historical analysis via Genie One, all under Unity Catalog governance. This eliminates separate data copies and synchronization pipelines, enabling risk monitoring and analytics to work from a single consistent data foundation.


### [How Shipsidekick replaced Neon with ClickHouse Managed Postgres for 10x faster queries and better reliability](https://yomu.fyi/post/how-shipsidekick-replaced-neon-with-clickhouse-managed-postgres-for-10.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Oct 23, 2026
- Type: Problem & solution

Shipsidekick, a logistics platform, replaced Neon with ClickHouse Managed Postgres after struggling with frequent outages, pool exhaustion, and slow support. The migration took a single weekend, with ClickPipes replicating 500 GB of data in 4-5 hours. Since the move, CPU usage dropped from 100% to 5-10%, cache hit rate rose from 30-35% to 99%, and queries run about 10x faster. The team resolved a partial unique index conflict during replication by dropping and re-creating the index. They also implemented timeouts on a read-only role to prevent AI agent queries from impacting primary performance.


### [The Fraud Analyst AI Coworker: Detecting Telecom Fraud in Real-Time on One Platform](https://yomu.fyi/post/the-fraud-analyst-ai-coworker-detecting-telecom-fraud-in-real-time-on.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Razi Bayati, Anthony Melnick
- Published: Oct 22, 2026
- Type: Problem & solution

Telecom fraud losses reached $41.82 billion in 2025, and each dollar of fraud costs carriers 3.4 times more when factoring chargebacks and overhead. Traditional detection systems rely on batch processing, leaving a gap attackers exploit within minutes through SIM swaps and account takeovers that surged 240% in Australia. Databricks addresses this by replacing the OLTP/OLAP split with Lakebase, a managed Postgres engine that syncs gold tables continuously for sub-millisecond feature retrieval, enabling real-time risk scoring against rules like impossible travel and roaming anomalies. An AI agent and Genie platform analyze historical incidents and automate investigations via tool-calling agents and a Knowledge Assistant trained on playbooks, shrinking the model adaptation cycle from days to near real time. The Solution Accelerator provides a complete pipeline from raw data to role-based workbenches for executives, managers, and analysts on a single governed platform.


### [Postgres at Open House NYC](https://yomu.fyi/post/postgres-at-open-house-nyc.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Oct 13, 2026
- Type: Announcement

At ClickHouse Open House NYC, the team announced WalShadow, an open-source engine that replicates Postgres into ClickHouse from the physical WAL, achieving sub-second latency and high throughput. They also launched ClickHouse Managed Postgres on Google Cloud in private preview, offering Postgres on local NVMe storage with native CDC into ClickHouse. The chdb Postgres extension was released, enabling fast imports from cloud storage using the embeddable ClickHouse engine. A live demo showed WalShadow keeping ClickHouse under 400 ms behind during a simulated trading platform with 50,000 trades per second, and detecting a fraud burst in under 500 ms compared to over 15 seconds with ClickPipes. The integrated stack aims to provide world-class Postgres for transactions and ClickHouse for analytics, with WalShadow as a key step toward unifying OLTP and OLAP.


### [How Fountain rebuilt its data plane on ClickHouse Cloud to power Cue, the Frontline Superintelligence](https://yomu.fyi/post/how-fountain-rebuilt-its-data-plane-on-clickhouse-cloud-to-power-cue-t.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Sep 22, 2026
- Type: Announcement

Fountain rebuilt its data plane on ClickHouse Cloud to power Cue, its agentic AI platform for frontline hiring, moving off a multi-vendor BigQuery and S3/Iceberg batch stack. The migration cut analytics latency from three hours to under two minutes, with most queries returning in under a second, at roughly 66% lower cost. Data from 13 Postgres production databases and four MongoDB databases streams into ClickHouse Cloud via ClickPipes managed CDC, landing as Iceberg and Parquet in S3 before reaching BigQuery and ClickHouse. The new architecture uses two symmetrical pipelines—one for Postgres, one for MongoDB—each running a short path: source, ClickPipes, materialized views, and Cue. The system now carries 8.48 billion CDC rows replicated continuously by ClickPipes and 10.3 billion analytics rows across 15 deployments, served by over 2,000 incremental materialized views.
