---
title: "Building a Service-oriented Architecture with Rails and Kafka"
description: "Service-oriented Rails applications can use Kafka to move high-volume events between producers and consumers without coupling downstream availability to an upstream request. Kafka persists events for a configured period, lets consumers manage offsets, and distributes replicated data across broker partitions, while ordering is guaranteed within a partition rather than across a topic. The post shows a Rails setup using the ruby-kafka gem, an asynchronous producer, JSON payloads, topics, and partition keys, then applies the event stream to order creation, email, and fulfillment workflows. It also addresses operational concerns including consumer scaling, lag metrics, duplicate delivery, idempotent processing with UPSERTs or event identifiers, and schema evolution, contrasting JSON’s accessibility with Avro’s stronger schema support."
---

# Building a Service-oriented Architecture with Rails and Kafka

[Heroku](https://yomu.fyi/company/heroku) · Stella Cotton · Jan 14, 2019

**Type:** Tutorial

## Summary

Service-oriented Rails applications can use Kafka to move high-volume events between producers and consumers without coupling downstream availability to an upstream request. Kafka persists events for a configured period, lets consumers manage offsets, and distributes replicated data across broker partitions, while ordering is guaranteed within a partition rather than across a topic. The post shows a Rails setup using the ruby-kafka gem, an asynchronous producer, JSON payloads, topics, and partition keys, then applies the event stream to order creation, email, and fulfillment workflows. It also addresses operational concerns including consumer scaling, lag metrics, duplicate delivery, idempotent processing with UPSERTs or event identifiers, and schema evolution, contrasting JSON’s accessibility with Avro’s stronger schema support.

## Context

Service-oriented applications may produce very large event volumes, while RPC-based designs require upstream services to monitor downstream availability, retry failures, and change when new downstream services are added. Event payload changes can also break consumers if schemas are not planned in advance.

## Approach / What changed

The post presents Kafka as an event-driven communication layer for Rails services. It uses the ruby-kafka gem with an asynchronous producer, topics, partitions, offsets, and partition keys, and discusses consumer scaling, lag monitoring, duplicate-event handling, and JSON or Avro payload formats.

## Takeaways

- Kafka consumers manage their own offsets, allowing them to resume from a specific event or read from the beginning or end of a stream.
- Kafka guarantees ordering within a partition, not across an entire topic; a partition\_key can keep related events in the same partition.
- At-least-once delivery requires consumers to tolerate duplicates, for example through UPSERT operations or unique event identifiers.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Kafka](https://yomu.fyi/topic/kafka), [Rails](https://yomu.fyi/topic/rails), [Ruby](https://yomu.fyi/topic/ruby)

- Source: [Heroku](https://www.heroku.com/blog/service-oriented-architecture-rails-kafka)
- Source URL: https://www.heroku.com/blog/service-oriented-architecture-rails-kafka
- Ingested by Yomu: 2026-09-09T14:26:59.242Z

[Read original post](https://www.heroku.com/blog/service-oriented-architecture-rails-kafka)