---
title: "Taking AUTO CDC to the next level: Solving the hardest real-world use cases"
description: "The post describes new AUTO CDC capabilities for challenges that standard CDC patterns do not fully address, including out-of-order bitemporal timelines, partial updates, and durable audit history. Bitemporal AUTO CDC tracks business and system time independently through four managed interval columns, rewriting affected history when corrections arrive out of order so either point-in-time view can be reconstructed. Partial Updates are generally available and treat selected incoming NULL values as \"do not update,\" preventing unchanged fields from overwriting stored values. The post also describes logging business and system timestamps in MLflow for reproducibility after VACUUM, and notes that the Python API for AUTO CDC Type 1 is being contributed to Apache Spark 4.2, with a SQL interface already merged for a next release."
---

# Taking AUTO CDC to the next level: Solving the hardest real-world use cases

[Databricks](https://yomu.fyi/company/databricks) · Josh Seidel, Shanelle Roman, Sudhanva Huruli · Aug 11, 2026

**Type:** Announcement

## Summary

The post describes new AUTO CDC capabilities for challenges that standard CDC patterns do not fully address, including out-of-order bitemporal timelines, partial updates, and durable audit history. Bitemporal AUTO CDC tracks business and system time independently through four managed interval columns, rewriting affected history when corrections arrive out of order so either point-in-time view can be reconstructed. Partial Updates are generally available and treat selected incoming NULL values as "do not update," preventing unchanged fields from overwriting stored values. The post also describes logging business and system timestamps in MLflow for reproducibility after VACUUM, and notes that the Python API for AUTO CDC Type 1 is being contributed to Apache Spark 4.2, with a SQL interface already merged for a next release.

## Context

Standard CDC patterns can struggle with out-of-order bitemporal timelines, updates that contain only changed fields, and audit or machine-learning reproducibility requirements that must survive storage retention and VACUUM.

## Approach / What changed

AUTO CDC adds dual-axis business-time and system-time history tracking, automatic handling for selected NULL values in partial updates, and open-source Apache Spark support. Bitemporal tables maintain managed intervals and rewrite affected history for out-of-order events, while MLflow timestamps can identify later training datasets.

## Takeaways

- Bitemporal AUTO CDC uses \_\_START\_AT and \_\_END\_AT for business time plus \_\_SYSTEM\_START\_AT and \_\_SYSTEM\_END\_AT for system time, allowing reconstruction along either timeline.
- AutoCDC Partial Updates is generally available and supports column lists, exceptions, or a per-row source column to determine which incoming NULL values mean "do not update."
- The contributed Apache Spark 4.2 Type 1 API includes state for early-arriving events, convergent retried microbatches, and compatibility with Delta Lake and Apache Iceberg.

**Tags:** [Data Pipelines](https://yomu.fyi/topic/data-pipelines), [Open Source](https://yomu.fyi/topic/open-source), [Reliability](https://yomu.fyi/topic/reliability), [Spark](https://yomu.fyi/topic/spark), [Streaming](https://yomu.fyi/topic/streaming)

- Source: [Databricks](https://www.databricks.com/blog/taking-auto-cdc-next-level-solving-hardest-real-world-use-cases)
- Source URL: https://www.databricks.com/blog/taking-auto-cdc-next-level-solving-hardest-real-world-use-cases
- Ingested by Yomu: 2026-08-30T16:51:35.735Z

[Read original post](https://www.databricks.com/blog/taking-auto-cdc-next-level-solving-hardest-real-world-use-cases)