Measuring Commercial Impact at Scale at Canva

Summary

Canva needed a scalable, standardized way to calculate the commercial impact of thousands of annual experiments on key business metrics like Monthly Active Users and Annual Recurring Revenue. Previously, disparate teams spent over six hours per experiment performing manual, error-prone calculations across fragmented spreadsheets and inconsistent data models. To resolve this, Canva built the IMPACT app using Snowflake, Streamlit, Snowpark, and Cortex to provide a self-serve platform tied directly to its central finance model. The application scales local uplift by actual audience exposure and supports pre-experiment scenario modeling alongside post-experiment tracking. A custom deployment workflow generates pull-request-isolated Streamlit environments in Snowflake stages, reducing time-to-insight to under ten minutes while enabling multiple developers to safely build and demo features in parallel.

Context

With over 1,800 experiments run annually across diverse product surfaces, measuring their downstream commercial impact on Monthly Active Users and Annual Recurring Revenue was chaotic. Impact estimation took over six hours per experiment, relying on manual calculations, disparate spreadsheets, varied assumptions, and fragmented data models across teams.

Approach / What changed

Canva built the IMPACT (Initiative Measurement and Personalization Analysis Comprehensive Toolkit) app on Snowflake, Streamlit, Snowpark, and Cortex. The tool aligns with the central finance model and properly scales uplift by user exposure. For development, a bash script provisions PR-isolated Snowflake stages and dedicated Streamlit instances for parallel testing.

Takeaways

• Uplifts observed on isolated product surfaces must be scaled by the specific subset of exposed users rather than applied globally to avoid overestimating company-level impact.
• The IMPACT app reduced commercial impact calculation time from more than six hours of manual analysis to under ten minutes.
• A custom development workflow deploys PR-specific Streamlit instances on Snowflake stages under a dedicated developer role, allowing engineers to test and demo isolated changes safely.

Related reading

• Canva · Feb 24, 2025

  Behind the scenes of Canva's DesignDNA campaign

  Canva launched DesignDNA in December 2024 as a personalized year-in-review campaign to highlight user achievements and showcase generative AI capabilities. Because internal privacy rules strictly prohibited inspecting personal designs, the engineering and creative teams inferred user preferences from the style and theme metadata tags on public templates. An initial keyword-matching algorithm paired 95% of users with one of seven emerging design trends, while generative AI keyword expansion increased coverage to 99%. The team also synthesized over one million localized poems and distinct design personalities using tools like Magic Write and Dream Lab. Finally, dynamic Canva template elements populated through URL parameters assembled 95 million distinct multi-page stories across nine locales.

  Divya Patel

• Canva · Jan 28, 2025

  Image replacement in Canva designs using reverse image search