---
title: "Using Propensity Score Matching to Uncover Shopify Capital’s Effect on Business Growth"
description: "Shopify’s data team sought to estimate whether accepting Shopify Capital for the first time increased merchants’ subsequent sales, without randomly denying funding in an A/B test. They compared US first-time adopters in January 2019 with Canadian shops that later pre-qualified for Capital, using propensity scores built from Shopify sales and performance characteristics and matching them with a caliper-based greedy nearest-neighbor algorithm. After matching about 600 treated shops against roughly 8,000 controls, they evaluated log-transformed cumulative GMV from February through July 2019 with a binary regression and checked balance using standardized mean differences, visual comparisons, and variance ratios. The estimated geometric-average GMV was 36% higher for US merchants, with a 95% confidence interval of 13% to 65%; robustness checks supported a positive treatment effect."
---

# Using Propensity Score Matching to Uncover Shopify Capital’s Effect on Business Growth

[Shopify](https://yomu.fyi/company/shopify) · 2023-10-18 · Nov 11, 2021

**Type:** Benchmark

## Summary

Shopify’s data team sought to estimate whether accepting Shopify Capital for the first time increased merchants’ subsequent sales, without randomly denying funding in an A/B test. They compared US first-time adopters in January 2019 with Canadian shops that later pre-qualified for Capital, using propensity scores built from Shopify sales and performance characteristics and matching them with a caliper-based greedy nearest-neighbor algorithm. After matching about 600 treated shops against roughly 8,000 controls, they evaluated log-transformed cumulative GMV from February through July 2019 with a binary regression and checked balance using standardized mean differences, visual comparisons, and variance ratios. The estimated geometric-average GMV was 36% higher for US merchants, with a 95% confidence interval of 13% to 65%; robustness checks supported a positive treatment effect.

## Context

The team wanted to determine whether Shopify Capital had a measurable effect on merchants’ future cumulative gross merchandise value. A randomized A/B test would have required automatically rejecting some interested merchants, which the team considered unacceptable. Canadian merchants served as an alternate comparison because Capital was not available there in 2019.

## Approach / What changed

The study used propensity score matching to compare US first-time Capital adopters in January 2019 with Canadian shops that later pre-qualified for Capital. Features based on Shopify sales and performance were used to estimate propensity scores, with many covariates logarithmically transformed. Caliper-based greedy nearest-neighbor matching produced balanced groups, followed by binary regression on log-transformed six-month GMV and robustness checks including an A/A simulation and bootstrapping.

## Takeaways

- The treatment group contained about 600 US shops adopting Capital for the first time in January 2019, while the control group contained approximately 8,000 Canadian shops that later pre-qualified for Capital.
- Caliper matching restricted nearest-neighbor matches to a defined maximum propensity-score distance, allowing unmatched shops rather than accepting distant comparisons; all but one treated shop found a Canadian counterpart.
- US merchants had a 36% higher geometric average of cumulative six-month GMV after first adopting Capital, with a 95% confidence interval ranging from 13% to 65%.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance)

- Source: [Shopify](https://shopify.engineering/propensity-score-matching-shopify-capital)
- Source URL: https://shopify.engineering/propensity-score-matching-shopify-capital
- Ingested by Yomu: 2026-08-30T15:28:34.332Z

[Read original post](https://shopify.engineering/propensity-score-matching-shopify-capital)