---
title: "How Heroku Operates its Multi-Tenant Apache Kafka Services"
description: "Heroku’s Apache Kafka service supports thousands of developers and millions of transactions, including customers sharing multi-tenant clusters. The post explains how those clusters provide security, predictable performance, availability, and production-like topology at lower cost than single-tenant plans. Multi-tenant and single-tenant offerings use the same numbers of brokers and ZooKeeper nodes, while Kafka ACLs, per-tenant resource namespaces, and producer/consumer byte quotas isolate customers and limit resource contention. Heroku provisions purchased resources, expands storage at 80% of quota, and uses higher defaults for partitions, replicas, and in-sync replicas; exceeding capacity can eventually throttle throughput without stopping the cluster. Automated tests model 50 users with varied traffic, exercise throughput and partition configurations, and simulate broker failures and restarts to identify operational limits and adjust the infrastructure before customer impact."
---

# How Heroku Operates its Multi-Tenant Apache Kafka Services

[Heroku](https://yomu.fyi/company/heroku) · Ali Hamidi · Jul 11, 2019

**Type:** Explainer

## Summary

Heroku’s Apache Kafka service supports thousands of developers and millions of transactions, including customers sharing multi-tenant clusters. The post explains how those clusters provide security, predictable performance, availability, and production-like topology at lower cost than single-tenant plans. Multi-tenant and single-tenant offerings use the same numbers of brokers and ZooKeeper nodes, while Kafka ACLs, per-tenant resource namespaces, and producer/consumer byte quotas isolate customers and limit resource contention. Heroku provisions purchased resources, expands storage at 80% of quota, and uses higher defaults for partitions, replicas, and in-sync replicas; exceeding capacity can eventually throttle throughput without stopping the cluster. Automated tests model 50 users with varied traffic, exercise throughput and partition configurations, and simulate broker failures and restarts to identify operational limits and adjust the infrastructure before customer impact.

## Context

Operating Kafka clusters for thousands of developers and millions of transactions requires maintaining capacity and uptime across varied customer use cases while keeping shared tenants secure, performant, and isolated. The service also needs to provide a production-like cluster structure at a lower cost than dedicated infrastructure.

## Approach / What changed

Heroku uses multi-broker, multi-ZooKeeper clusters with Kafka ACLs, per-tenant resource namespaces, and producer and consumer quotas. It provisions and expands storage, applies configuration defaults for partitions and replication, and validates capacity and failure behavior through automated tests involving 50 simulated users, varied traffic profiles, broker failures, and restarts.

## Takeaways

- Kafka ACLs restrict which users can perform actions on specific topics, while per-tenant resource namespaces associate resources with an account and application.
- Producer and consumer byte-per-second quotas limit individual tenants’ resource usage so heavy activity from one customer does not degrade service for others.
- Automated tests vary user traffic, throughput limits, and partition counts, then simulate broker outages and restarts to determine operational limits and verify failover.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [High Availability](https://yomu.fyi/topic/high-availability), [Kafka](https://yomu.fyi/topic/kafka), [Testing](https://yomu.fyi/topic/testing)

- Source: [Heroku](https://www.heroku.com/blog/how-heroku-operates-its-multi-tenant-apache-kafka-services)
- Source URL: https://www.heroku.com/blog/how-heroku-operates-its-multi-tenant-apache-kafka-services
- Ingested by Yomu: 2026-09-09T14:27:36.415Z

[Read original post](https://www.heroku.com/blog/how-heroku-operates-its-multi-tenant-apache-kafka-services)