Image replacement in Canva designs using reverse image search

Summary

Canva needed an automated way to replace media in design templates, such as when third-party licensing partnerships expire across more than 150 million images. Existing recommendation engines, perceptual hashing, and text metadata searches failed to capture visual similarity hierarchies or ensure replacement relevance. To build a reverse image search system, engineers evaluated embedding models including CLIP, ViTMAE, DreamSim, CaiT, and DINOv2 alongside an external vector database supporting metadata filtering. Evaluation on sample datasets identified DINOv2 as the best model for preserving subjects, background context, and color tones in photos. Integrated into the Template Assistant as a human-in-the-loop tool, the automated suggestions increased image replacement speeds by 4.5 times during initial pilot testing.

Context

Replacing media in Canva templates when third-party partnerships expire was a lengthy, manual process requiring an automated solution to find intellectual property-safe, visually similar replacements across a 150-million-image library while preserving aspect ratios and visual features.

Approach / What changed

Evaluated multiple computer vision embedding models (DINOv2, CLIP, ViTMAE, DreamSim, CaiT, and GPT-4o image descriptions paired with CLIP) using Faiss on a 50,000-image dataset. Chose DINOv2 paired with a third-party external vector database to support real-time updates and metadata filtering, integrating the top eight similar image suggestions into the Template Assistant UI for human review.

Takeaways

• DINOv2 was selected as the most effective embedding model for visual similarity in photos, outperforming alternatives like CLIP, DreamSim, and GPT-4o-generated descriptions.
• An external vector database was chosen over an in-memory Faiss index to avoid large RAM hardware costs, scale with media library changes, and enable metadata filtering like aspect ratios.
• The system achieved a 4.5x speedup in template image replacement during pilots, though it performed better on photos than on graphics, cartoons, or images with text and symbols.

Related reading

• Canva · Jun 20, 2025

  Measuring Commercial Impact at Scale at Canva

  Canva needed a scalable, standardized way to calculate the commercial impact of thousands of annual experiments on key business metrics like Monthly Active Users and Annual Recurring Revenue. Previously, disparate teams spent over six hours per experiment performing manual, error-prone calculations across fragmented spreadsheets and inconsistent data models. To resolve this, Canva built the IMPACT app using Snowflake, Streamlit, Snowpark, and Cortex to provide a self-serve platform tied directly to its central finance model. The application scales local uplift by actual audience exposure and supports pre-experiment scenario modeling alongside post-experiment tracking. A custom deployment workflow generates pull-request-isolated Streamlit environments in Snowflake stages, reducing time-to-insight to under ten minutes while enabling multiple developers to safely build and demo features in parallel.

  Jun Ye

• Canva · Feb 24, 2025

  Behind the scenes of Canva's DesignDNA campaign