Loading post…
ShopifyKrystian CzesakBenchmark
Shaping the Future of Payments in the Browser
Summary
Shopify tested whether the Payment Request API was ready for its merchants and buyers on supported browsers. The team defined checkout completion rate, time to purchase, drop-off, and qualitative checkout experience as success measures, then ran an A/B experiment with selected merchants and eligible buyers. The isolated Ruby app injected storefront JavaScript, checked eligibility and credit-card support, and routed buyers to either the regular checkout or a browser-provided payment sheet. The experiment recorded a 7% checkout conversion rate drop and a 60% drop at the payment sheet, although time to completion medians were promising. Shopify concluded Payment Request was not yet a good fit, while expecting greater familiarity, payment-sheet improvements, and third-party payment methods through Payment Handler API to help adoption.
Context
Shopify participated in the W3C Web Payments Working Group and tested Payment Request while browser and developer adoption were still early. The experiment focused on supported browsers and credit-card transactions because basic-card was the widely implemented payment method at the time.
Approach / What changed
Shopify built an isolated Ruby app using its existing Rails app infrastructure and APIs, deployed it to Google Cloud, and injected storefront JavaScript through an installed merchant app. Eligible buyers were split evenly between a regular-checkout control group and a Payment Request experiment group that saw the browser payment sheet.
Takeaways
The experiment split eligible buyers 50/50 between the regular checkout and a Payment Request flow, with eligibility dependent on browser support and available credit-card payment capability.
Checkout conversion fell by 7%, and 60% of buyers dropped off at the payment sheet; however, the reported median times to completion were promising.
Shopify identified payment-sheet usability issues, including unclear calls to action, confusing unselected grayed-out fields, and validation errors that appeared too late.
FAQ
How were buyers assigned to the Payment Request experiment?
Eligible buyers were assigned evenly: 50% entered a control group using Shopify’s regular checkout, and 50% entered the experiment group and saw the browser payment sheet. Eligibility required a supported browser, while the participating merchants needed to accept direct credit-card payments.
What happened to checkout performance when buyers saw the payment sheet?
Checkout conversion dropped by 7%, and 60% of buyers dropped at the payment sheet. The experiment also found promising median times to completion, so the results were mixed rather than uniformly negative.
Which payment-sheet design issues did Shopify identify?
Shopify reported that buyers could be unsure how to proceed when calls to action changed position or when grayed-out fields were shown without being selected. It recommended keeping negative actions in a consistent location, avoiding misleading inactive fields, and showing validation errors near the relevant inputs as early as possible.
Source and freshness
- Source
- shopify.engineering
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.