---
title: "Read, don't copy: replacing file and API feeds with OpenSharing - Part 1 (The Why)"
description: "Traditional customer data ingestion via SFTP, APIs, or managed connectors carries hidden costs like pipeline drift, backfill complexity, and duplicate storage. OpenSharing is an open protocol built on table formats such as Delta Lake and Iceberg that lets consumers read a customer’s table in place instead of copying it. By exposing a change data feed, OpenSharing enables incremental pulls of only new, updated, or deleted records without manual timestamp tracking or per-customer connector babysitting. Backfills become simple queries against the shared historical table, eliminating the need to ask customers to re-export data or hit API rate limits. Customers benefit by sharing tables they already maintain, gaining an audit trail of access via Databricks system tables, and avoiding export babysitting, while consumers use their own compute and avoid duplicate storage costs."
---

# Read, don't copy: replacing file and API feeds with OpenSharing - Part 1 (The Why)

[Databricks](https://yomu.fyi/company/databricks) · Jordan, Soldo, Nikhil, Tallam · Nov 7, 2026

**Type:** Explainer

## Summary

Traditional customer data ingestion via SFTP, APIs, or managed connectors carries hidden costs like pipeline drift, backfill complexity, and duplicate storage. OpenSharing is an open protocol built on table formats such as Delta Lake and Iceberg that lets consumers read a customer’s table in place instead of copying it. By exposing a change data feed, OpenSharing enables incremental pulls of only new, updated, or deleted records without manual timestamp tracking or per-customer connector babysitting. Backfills become simple queries against the shared historical table, eliminating the need to ask customers to re-export data or hit API rate limits. Customers benefit by sharing tables they already maintain, gaining an audit trail of access via Databricks system tables, and avoiding export babysitting, while consumers use their own compute and avoid duplicate storage costs.

## Context

The post discusses the hidden costs of traditional customer data ingestion patterns (SFTP, APIs, managed connectors) and introduces OpenSharing, an open protocol for reading tables in place, as a solution. It is Part 1 of a series, focusing on the ‘why’ before the ‘how’ in Part 2.

## Takeaways

- OpenSharing uses open table formats like Delta Lake and Iceberg to let consumers read a customer’s table in place, eliminating the need for file exports or API pulls.
- By exposing a change data feed, OpenSharing enables incremental data pulls that track only new, updated, or deleted records, avoiding manual timestamp bookkeeping.
- Customers gain an audit trail of data access through Databricks system tables, replacing blind trust with verifiable logs.

## FAQ

### How does OpenSharing enable incremental data pulls?

OpenSharing exposes a change data feed on shared tables. When your pipeline runs, it reads the feed to know which records were created, deleted, or updated, so you only pull changes instead of reprocessing entire files.

### How does OpenSharing simplify backfills?

If the customer shares the entire historical table, you can query the data you need to backfill directly. You do not need to ask the customer to re-export data or hit API rate limits, because you have direct access to the table.

### What audit capabilities does OpenSharing provide to customers?

With Databricks system tables, customers can see when a recipient queries a shared table. This replaces blind trust with a verifiable audit trail of every access.

**Tags:** [Data Sharing](https://yomu.fyi/topic/data-sharing), [Delta Lake](https://yomu.fyi/topic/delta-lake), [Iceberg](https://yomu.fyi/topic/iceberg), [OpenSharing](https://yomu.fyi/topic/open-sharing)

- Source: [Databricks](https://www.databricks.com/blog/read-dont-copy-replacing-file-and-api-feeds-opensharing-part-1-why)
- Source URL: https://www.databricks.com/blog/read-dont-copy-replacing-file-and-api-feeds-opensharing-part-1-why
- Ingested by Yomu: 2026-11-07T00:00:14.882Z

[Read original post](https://www.databricks.com/blog/read-dont-copy-replacing-file-and-api-feeds-opensharing-part-1-why)
