Topic

Cluster Mirroring

1 post about Cluster Mirroring. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Posts

Red Hat
Data liberation: Apache Kafka's native cluster mirroring
KIP-1279 introduces native cluster mirroring, embedding cross-cluster replication directly into the Kafka broker. This eliminates external tooling, offset translation tables, and recompression, as destination brokers fetch committed records using the standard fetch protocol and append them byte for byte. The approach preserves offsets, compression, and consumer group state, simplifying failover to stopping the mirror and redirecting clients. The architecture uses mirror fetcher threads inside the broker, extending AbstractFetcherThread, and leverages existing client quota mechanisms. Migration involves creating a new KRaft cluster, setting up a mirror, and starting mirroring, with future work including synchronous mirroring and tiered storage integration.

The best engineering reads, once a week.

Topic

1 post about Cluster Mirroring. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Red Hat
Data liberation: Apache Kafka's native cluster mirroring
KIP-1279 introduces native cluster mirroring, embedding cross-cluster replication directly into the Kafka broker. This eliminates external tooling, offset translation tables, and recompression, as destination brokers fetch committed records using the standard fetch protocol and append them byte for byte. The approach preserves offsets, compression, and consumer group state, simplifying failover to stopping the mirror and redirecting clients. The architecture uses mirror fetcher threads inside the broker, extending AbstractFetcherThread, and leverages existing client quota mechanisms. Migration involves creating a new KRaft cluster, setting up a mirror, and starting mirroring, with future work including synchronous mirroring and tiered storage integration.

The best engineering reads, once a week.