The foundations of Canva’s continuous data platform with Snowpipe Streaming

Summary

As Canva expanded to over 200 million monthly active users, ingestion throughput reached 25 billion records daily, causing AWS Data Firehose costs to consume nearly half of the product analytics platform budget. To reduce these expenses and eliminate intermediate file staging, the engineering team integrated Snowflake's Snowpipe Streaming directly with their Java-based Kinesis Data Streams pipeline. The architecture streams records directly into Snowflake tables using logical channels with offset checkpoints while configuring client buffering up to a five-minute maximum lag. In handling Kinesis Client Library edge cases, record processors coordinate with the LeaseCoordinator to drop leases when channels become unhealthy. Operating in production for over six months, the system ingested more than 20.35 petabytes of data, decreased query latency to roughly ten minutes, and reduced overall cloud spend by 45 percent.

Context

Canva's user base grew to over 200 million monthly active users, generating 25 billion records daily. Ingesting this data into Snowflake via AWS Data Firehose grew to account for nearly 50% of the platform's cost due to Firehose's 5 KB billing minimums and incremental charges, as well as cost increases from using Kinesis Data Streams as a source.

Approach / What changed

Canva replaced the Firehose and S3 staging pipeline by integrating Kinesis Data Streams directly with Snowflake's Snowpipe Streaming API in Java workers. Snowpipe Streaming ingests rows directly into Snowflake tables via channels tracked with offset tokens. To optimize cost, Canva buffered data using a MAX_CLIENT_LAG setting of 5 minutes and handled channel invalidation failures by dropping Kinesis Client Library leases via LeaseCoordinator.

Takeaways

• AWS Data Firehose bills in 5 KB increments with a 5 KB minimum size per record for S3 destinations, which significantly drove up costs at Canva's 25-billion-record daily volume.
• Snowpipe Streaming charges 0.01 credits per client per hour billed in 1-second increments plus intermediate file migration compute, while host-to-S3 transfers avoid extra data transfer costs.
• Because the Kinesis Client Library lacks a native restart mechanism when a Snowpipe Streaming channel fails, Canva allows record processors to declare unhealthiness and drop leases via LeaseCoordinator.

Related reading

• Grab · Jan 31, 2024

  Rethinking Stream Processing: Data Exploration

  Online data materialised in Kafka topics at Grab traditionally remained uninspected until ingested into the Data Lake as offline data, introducing latency and making stream processing applications difficult to develop. To enable earlier data exploration, Grab adopted Apache Zeppelin notebooks integrated with an internal data streaming platform. Zeppelin uses a Flink interpreter to convert interactive SQL queries into Flink jobs submitted to a Flink session cluster. The platform dynamically translates Protobuf schemas into SQL Data Definition Language statements to simplify table discovery. Security and compliance risks are addressed by combining mutual TLS authentication with Strimzi's Open Policy Agent to enforce read-only access policies and audit logging on Kafka topics.

  Shi Kai Ng

• Canva · Jun 20, 2025

  Measuring Commercial Impact at Scale at Canva

  Canva needed a scalable, standardized way to calculate the commercial impact of thousands of annual experiments on key business metrics like Monthly Active Users and Annual Recurring Revenue. Previously, disparate teams spent over six hours per experiment performing manual, error-prone calculations across fragmented spreadsheets and inconsistent data models. To resolve this, Canva built the IMPACT app using Snowflake, Streamlit, Snowpark, and Cortex to provide a self-serve platform tied directly to its central finance model. The application scales local uplift by actual audience exposure and supports pre-experiment scenario modeling alongside post-experiment tracking. A custom deployment workflow generates pull-request-isolated Streamlit environments in Snowflake stages, reducing time-to-insight to under ten minutes while enabling multiple developers to safely build and demo features in parallel.