---
title: "Filter, reshape, and redact logs before export with Transformers"
description: "Cloudflare Logpush now supports Transformers, allowing teams to filter, redact, and reshape log streams using SQL before export to SIEMs, analytics platforms, or object storage. This eliminates the need for a separate processing service that teams would otherwise have to deploy, scale, and maintain. Transformers are SQL queries applied per batch within Logpush’s high-throughput pipeline, using Apache DataFusion and Arrow for columnar execution. The implementation embeds DataFusion in Bufstream, the Go-based delivery service, using cgo and Arrow IPC to pass typed batches across the language boundary without introducing network hops. Customers can create, preview, and version Transformers via the dashboard or API, and attach them to NDJSON Logpush jobs to reduce downstream data volume and cost."
---

# Filter, reshape, and redact logs before export with Transformers

[Cloudflare](https://yomu.fyi/company/cloudflare) · Sahidya Devadoss · Oct 12, 2026

**Type:** Announcement

## Summary

Cloudflare Logpush now supports Transformers, allowing teams to filter, redact, and reshape log streams using SQL before export to SIEMs, analytics platforms, or object storage. This eliminates the need for a separate processing service that teams would otherwise have to deploy, scale, and maintain. Transformers are SQL queries applied per batch within Logpush’s high-throughput pipeline, using Apache DataFusion and Arrow for columnar execution. The implementation embeds DataFusion in Bufstream, the Go-based delivery service, using cgo and Arrow IPC to pass typed batches across the language boundary without introducing network hops. Customers can create, preview, and version Transformers via the dashboard or API, and attach them to NDJSON Logpush jobs to reduce downstream data volume and cost.

## Context

Cloudflare Logpush now includes Transformers, a SQL-based feature that allows teams to filter, redact, and reshape log records before export, eliminating the need for separate processing services.

## Approach / What changed

Embedding Apache DataFusion, a Rust SQL engine, into Bufstream (the Go-based Logpush delivery service) using cgo and Arrow IPC to execute SQL transformations in-process without network hops.

## Takeaways

- Transformers use SQL queries to filter, redact, and reshape logs before export, reducing downstream data volume and cost.
- The implementation embeds Apache DataFusion in Bufstream using cgo and Arrow IPC, avoiding network calls and maintaining high throughput.
- Customers can create, preview, and version Transformers via the Cloudflare dashboard or API, and attach them to NDJSON Logpush jobs.

## FAQ

### How do Transformers work?

A Transformer is a SQL query written for a specific Logpush dataset. Logpush applies the query to each batch before export, removing records that do not match the WHERE clause and exporting the output of the SELECT statement in NDJSON format.

### How is the SQL executed?

Bufstream executes the SQL query using Apache DataFusion embedded in-process. It passes typed Arrow batches across the Go-Rust boundary via cgo and Arrow IPC, avoiding network calls and maintaining high throughput.

### What happens if a transform fails?

If a transform fails, Bufstream does not advance the delivery offset or send untransformed data. It retries with progressive delays and eventually disables the job if it remains in a failing state.

**Tags:** [Apache Arrow](https://yomu.fyi/topic/apache-arrow), [Apache DataFusion](https://yomu.fyi/topic/apache-datafusion), [Logpush](https://yomu.fyi/topic/logpush), [SQL](https://yomu.fyi/topic/sql), [Transformers](https://yomu.fyi/topic/transformers)

- Source: [Cloudflare](https://blog.cloudflare.com/transformers-on-logpush)
- Source URL: https://blog.cloudflare.com/transformers-on-logpush
- Ingested by Yomu: 2026-10-12T15:00:12.411Z

[Read original post](https://blog.cloudflare.com/transformers-on-logpush)