---
title: "Building a Real-time Buyer Signal Data Pipeline for Shopify Inbox"
description: "Shopify built a real-time buyer-signal pipeline for Shopify Inbox after noting that 70 percent of its conversations involved customers making a purchasing decision. The system surfaces cart actions and order-completion information as chronological conversation events, covering events from up to 14 days before a conversation and post-conversation events while its seven-day lifecycle remains active. It uses Kafka through Monorail and Change Data Capture, Apache Beam on Google Cloud Dataflow, filtering jobs for mission-critical transactional events, and stateful aggregation with timers and a Global Window to retain buyer activity. In an equal-group A/B test, buyer signals increased merchant response rate by two percentage points and attributed conversion rate by 0.7 percentage points, while response time showed no significant change."
---

# Building a Real-time Buyer Signal Data Pipeline for Shopify Inbox

[Shopify](https://yomu.fyi/company/shopify) · 2023-10-18 · Dec 13, 2021

**Type:** Problem & solution

## Summary

Shopify built a real-time buyer-signal pipeline for Shopify Inbox after noting that 70 percent of its conversations involved customers making a purchasing decision. The system surfaces cart actions and order-completion information as chronological conversation events, covering events from up to 14 days before a conversation and post-conversation events while its seven-day lifecycle remains active. It uses Kafka through Monorail and Change Data Capture, Apache Beam on Google Cloud Dataflow, filtering jobs for mission-critical transactional events, and stateful aggregation with timers and a Global Window to retain buyer activity. In an equal-group A/B test, buyer signals increased merchant response rate by two percentage points and attributed conversion rate by 0.7 percentage points, while response time showed no significant change.

## Context

Most Shopify Inbox conversations involved customers making a purchasing decision, but merchants needed relevant purchase-intent signals to understand where buyers were in their shopping journeys, answer questions more directly, and prioritize conversations likely to convert.

## Approach / What changed

The pipeline ingests cart, checkout, and conversation data from Kafka sources including Monorail and Change Data Capture, filters transactional events, and uses Apache Beam stateful processing, timers, and a Global Window to aggregate buyer activity. Google Cloud Dataflow runs the Beam model, which emits cart and order-completion context through Monorail for downstream delivery as conversation events.

## Takeaways

- Change Data Capture uses MySQL binlogs and Debezium to turn data that is not naturally streamed into a stream of events, while Monorail provides structured, versioned Kafka events.
- The aggregation job keys conversation, checkout, and cart inputs by the buyer identifier and retains historical state in a Global Window, supporting access to prior events and an extendable cart lifespan.
- The A/B test found a two-percentage-point increase in response rate and a 0.7-percentage-point increase in attributed conversion rate, with no significant change in response time.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Data Pipelines](https://yomu.fyi/topic/data-pipelines), [Kafka](https://yomu.fyi/topic/kafka), [Testing](https://yomu.fyi/topic/testing)

- Source: [Shopify](https://shopify.engineering/real-time-buyer-signal-data-pipeline-shopify-inbox)
- Source URL: https://shopify.engineering/real-time-buyer-signal-data-pipeline-shopify-inbox
- Ingested by Yomu: 2026-08-30T15:28:16.673Z

[Read original post](https://shopify.engineering/real-time-buyer-signal-data-pipeline-shopify-inbox)