# Image replacement in Canva designs using reverse image search

[Canva](https://yomu.fyi/company/canva) · Sam Jacobs · Jan 28, 2025

**Type:** Problem & solution

## Summary

Canva needed an automated way to replace media in design templates, such as when third-party licensing partnerships expire across more than 150 million images. Existing recommendation engines, perceptual hashing, and text metadata searches failed to capture visual similarity hierarchies or ensure replacement relevance. To build a reverse image search system, engineers evaluated embedding models including CLIP, ViTMAE, DreamSim, CaiT, and DINOv2 alongside an external vector database supporting metadata filtering. Evaluation on sample datasets identified DINOv2 as the best model for preserving subjects, background context, and color tones in photos. Integrated into the Template Assistant as a human-in-the-loop tool, the automated suggestions increased image replacement speeds by 4.5 times during initial pilot testing.

## Context

Replacing media in Canva templates when third-party partnerships expire was a lengthy, manual process requiring an automated solution to find intellectual property-safe, visually similar replacements across a 150-million-image library while preserving aspect ratios and visual features.

## Approach / What changed

Evaluated multiple computer vision embedding models (DINOv2, CLIP, ViTMAE, DreamSim, CaiT, and GPT-4o image descriptions paired with CLIP) using Faiss on a 50,000-image dataset. Chose DINOv2 paired with a third-party external vector database to support real-time updates and metadata filtering, integrating the top eight similar image suggestions into the Template Assistant UI for human review.

## Takeaways

- DINOv2 was selected as the most effective embedding model for visual similarity in photos, outperforming alternatives like CLIP, DreamSim, and GPT-4o-generated descriptions.
- An external vector database was chosen over an in-memory Faiss index to avoid large RAM hardware costs, scale with media library changes, and enable metadata filtering like aspect ratios.
- The system achieved a 4.5x speedup in template image replacement during pilots, though it performed better on photos than on graphics, cartoons, or images with text and symbols.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Search](https://yomu.fyi/topic/search)

[Read original post](https://www.canva.dev/blog/engineering/image-replacement-in-canva-designs-using-reverse-image-search)