Loading post…
ShopifyCam Davidson-PilonExplainer
How Shopify Merchants can Measure Retention
Summary
Shopify addresses customer retention for merchants operating non-contractual businesses, where churn is not directly observable. Common rules that label customers inactive after N days create arbitrary binary states, so the post proposes probabilistic modeling with the BG/NBD (Beta Geometric / Negative Binomial Distribution) model. It represents each customer with a purchase rate and churn probability, estimates their population distributions, and uses age, recency, and repeat-purchase frequency as sufficient statistics. Shopify fits a modified implementation in lifetimes using Apache Spark DataFrames, subsampling up to 50k customers per shop for performance, and produces activity probabilities and expected future purchases for more than 500 million historical merchant customers. The model is interpretable and supports customer ranking and targeted campaigns, but it handles seasonality poorly and does not easily incorporate additional variables such as country or products purchased.
Context
Non-contractual businesses do not receive a direct notification when a customer churns. The post contrasts probabilistic churn estimation with fixed time-limit rules that reduce customers to arbitrary active or inactive states.
Approach / What changed
The BG/NBD model treats each customer as having a purchase rate and a churn probability, with those parameters drawn from Gamma and Beta distributions. Its likelihood uses age, recency, and repeat-purchase frequency; Shopify estimates the distribution parameters with optimization, applies a modified implementation in lifetimes, and scales it with Apache Spark DataFrames and Python UDFs. It ranks customers by probability of being active and calculates expected future purchases.
Takeaways
The BG/NBD model uses age, recency, and repeat-purchase frequency as sufficient statistics for estimating a customer’s probability of being active.
Shopify trains the model across more than 500,000 merchants, subsampling to 50,000 customers per shop because larger samples produced diminishing returns.
The model is interpretable but does not handle seasonal purchasing well and cannot easily add variables such as country or products purchased.
FAQ
Why is defining churn as no purchase after a fixed number of days problematic?
A fixed cutoff creates only active and inactive states, even though customers with different purchase habits can have very different likelihoods of returning. A weekly bulk buyer and a daily buyer should not necessarily be treated the same after the same period.
What data does the BG/NBD model need for each customer?
The model uses age, recency, and frequency. These are computed from raw purchase data, representing the time since the first purchase, the age at the last purchase, and the number of repeat purchases.
How does Shopify scale the model across its merchant base?
Apache Spark DataFrames group raw purchase rows by shop and apply a Python user-defined function to each partition. Shopify fits the model, applies it to each customer, and infers churn probabilities and expected values for more than 500 million historical merchant customers.
Source and freshness
- Source
- shopify.engineering
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.