---
title: "Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces"
description: "Lyft describes a framework for estimating long-term effects of pricing and incentive decisions in its two-sided marketplace, where direct and market-mediated consequences can diverge. The method first uses residualized observational regressions to estimate how policy changes alter distributions of negative experiences such as wait times, surge, cancellations, idleness, and hourly earnings, validating those responses with switch-back experiments. It then applies Augmented Inverse Probability Weighting to estimate how these experiences affect future outcomes, summarizes the relationship with a surrogacy index, and checks calibration using user-split experiments. Finally, Lyft combines mediated and separately estimated direct effects and uses region-split experiments for end-to-end validation, selecting treated and control regions with a forward-selection algorithm inspired by FDiD. The framework is intended to support scenario planning, budget allocation, and continuous calibration of long-run marketplace forecasts."
---

# Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces

[Lyft](https://yomu.fyi/company/lyft) · Iraklikhorguani · Mar 25, 2026

**Type:** Explainer

## Summary

Lyft describes a framework for estimating long-term effects of pricing and incentive decisions in its two-sided marketplace, where direct and market-mediated consequences can diverge. The method first uses residualized observational regressions to estimate how policy changes alter distributions of negative experiences such as wait times, surge, cancellations, idleness, and hourly earnings, validating those responses with switch-back experiments. It then applies Augmented Inverse Probability Weighting to estimate how these experiences affect future outcomes, summarizes the relationship with a surrogacy index, and checks calibration using user-split experiments. Finally, Lyft combines mediated and separately estimated direct effects and uses region-split experiments for end-to-end validation, selecting treated and control regions with a forward-selection algorithm inspired by FDiD. The framework is intended to support scenario planning, budget allocation, and continuous calibration of long-run marketplace forecasts.

## Context

Estimating long-term effects of resource allocation decisions is difficult in a multi-sided marketplace because policy changes create direct effects and indirect market-mediated effects. Changes to incentives, prices, supply, or demand can alter user experiences and later behavior across riders and drivers, while seasonal patterns and changing market conditions complicate observational estimation.

## Approach / What changed

The framework uses three connected stages. Residualized observational regressions estimate policy effects on distributions of negative user experiences and are checked with switch-back experiments. AIPW estimates how those experiences affect future outcomes through a surrogacy index, with user-split experiments used for calibration. Direct and mediated long-term effects are then combined and tested with region-split experiments whose treated and control regions are selected using forward selection inspired by FDiD.

## Takeaways

- Residualizing time-of-week patterns and controlling for supply and demand lets policy coefficients represent incremental changes in negative user experiences around normal operating conditions.
- AIPW combines exposure-propensity and outcome models to estimate how negative experiences affect future outcomes while controlling for time, location, and user history.
- Region-split experiments provide end-to-end validation, but the design requires forward selection to improve pre-intervention fit and expected statistical power.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Performance](https://yomu.fyi/topic/performance)

- Source: [Lyft](https://eng.lyft.com/beyond-a-b-testing-using-surrogacy-and-region-splits-to-measure-long-term-effects-in-marketplaces-9cb06d628f2d)
- Source URL: https://eng.lyft.com/beyond-a-b-testing-using-surrogacy-and-region-splits-to-measure-long-term-effects-in-marketplaces-9cb06d628f2d
- Ingested by Yomu: 2026-08-27T15:03:39.314Z

[Read original post](https://eng.lyft.com/beyond-a-b-testing-using-surrogacy-and-region-splits-to-measure-long-term-effects-in-marketplaces-9cb06d628f2d)