---
title: "Fast Copy-On-Write within Apache Parquet for Data Lakehouse ACID Upserts"
description: "Large-scale lakehouse upserts can become slow and resource-intensive, especially in Apache Hudi copy-on-write mode, where affected files are read, modified, and rewritten in full. The approach adds a Parquet row-level secondary index mapping records to files and row IDs, allowing updates to locate the specific data pages containing changed records. During copy-on-write, related pages are updated while unrelated pages are copied as byte buffers, avoiding decompression, re-encoding, and record reassembly for skipped data. Benchmarks using TPC-DS sales data, Spark jobs with matching vCore and memory settings, and update rates from 5% to 50% found significantly faster speeds than traditional approaches, including Delta Lake’s default configuration. The authors plan to integrate the row-level index and fast copy-on-write into Apache Hudi for use cases such as incremental ingestion, data deletion, and CDC."
---

# Fast Copy-On-Write within Apache Parquet for Data Lakehouse ACID Upserts

[Uber](https://yomu.fyi/company/uber) · May 12, 2026

**Type:** Benchmark

## Summary

Large-scale lakehouse upserts can become slow and resource-intensive, especially in Apache Hudi copy-on-write mode, where affected files are read, modified, and rewritten in full. The approach adds a Parquet row-level secondary index mapping records to files and row IDs, allowing updates to locate the specific data pages containing changed records. During copy-on-write, related pages are updated while unrelated pages are copied as byte buffers, avoiding decompression, re-encoding, and record reassembly for skipped data. Benchmarks using TPC-DS sales data, Spark jobs with matching vCore and memory settings, and update rates from 5% to 50% found significantly faster speeds than traditional approaches, including Delta Lake’s default configuration. The authors plan to integrate the row-level index and fast copy-on-write into Apache Hudi for use cases such as incremental ingestion, data deletion, and CDC.

## Context

Apache Hudi, Delta Lake, and Apache Iceberg support lakehouse use cases such as incremental ingestion, data retention, and change data capture. Copy-on-write can become costly as data volume grows because files containing updated records are rewritten, and the source cites cases where updates affected 90 percent of files and involved around 100 TB of rewrites.

## Approach / What changed

The implementation introduces a Parquet row-level secondary index that maps a record to its file and row ID, then uses Parquet metadata to identify the affected data pages. Pages in scope are updated, while unrelated pages are copied byte-for-byte into the new file without decompression, re-encoding, or record reassembly.

## Takeaways

- The row-level secondary index maps records to file and row ID, enabling updates to locate affected Parquet pages instead of only locating files.
- Partial copy-on-write updates only related data pages and copies unrelated pages as byte buffers, avoiding expensive decompression, re-encoding, and record reassembly.
- TPC-DS benchmarking with 5% to 50% of data updated reported significantly faster speeds than traditional approaches, including Delta Lake with default configurations.

## FAQ

### How does the row-level index reduce copy-on-write work?

It maps each record to a file and row ID, which identifies the Parquet data pages containing the records to update. Other pages can then be skipped during modification and copied unchanged.

### What does partial copy-on-write do with unrelated pages?

It copies unrelated pages as byte buffers into the new file without decompressing, re-encoding, or reassembling their records. Only pages within the update scope are changed.

### How was the performance comparison conducted?

The tests used TPC-DS sales data and Spark jobs with the same vCore and memory settings. They compared update rates from 5% to 50% against Delta Lake using its default out-of-box configuration.

**Tags:** [Apache Hudi](https://yomu.fyi/topic/apache-hudi), [Lakehouse](https://yomu.fyi/topic/lakehouse), [Parquet](https://yomu.fyi/topic/parquet)

- Source: [Uber](https://www.uber.com/us/en/blog/fast-copy-on-write-within-apache-parquet)
- Source URL: https://www.uber.com/us/en/blog/fast-copy-on-write-within-apache-parquet
- Ingested by Yomu: 2026-09-14T08:02:02.956Z

[Read original post](https://www.uber.com/us/en/blog/fast-copy-on-write-within-apache-parquet)