# Image search with 🤗 datasets

huggingface.co · Daniel van Strien · Mar 16, 2022

**Type:** Tutorial

## Summary

Hugging Face datasets expanded its capabilities by introducing an Image feature type, enabling image processing and integration with vector indexing tools. The library was applied to a sample of historical book embellishments extracted via OCR from the British Library. Images were loaded using the ImageFolder loader, enriched with filename metadata, and pushed to the Hugging Face Hub. A FAISS index and CLIP embeddings were used to retrieve images matching natural language queries, such as categories, specific objects, and boolean operators. While the retrieval system demonstrated reasonable semantic search results across various prompts, full public deployment was avoided due to CLIP model card restrictions and potential bias in the historical dataset.

## Context

Historical book illustrations from the British Library had known source book metadata but lacked rich semantic descriptions of their visual content, limiting search capabilities.

## Approach / What changed

Loaded the image collection using the datasets ImageFolder loader, extracted filename metadata into a new column, pushed the dataset to the Hugging Face Hub, and indexed CLIP embeddings with FAISS for text-to-image semantic retrieval.

## Takeaways

- The datasets Image feature accepts absolute file paths, byte dictionaries, NumPy arrays, or PIL Image objects.
- The ImageFolder loader allows direct ingestion of directory structures containing images into a DatasetDict.
- Public deployment of the search demo was withheld due to CLIP out-of-scope deployment guidance and bias concerns in historical source books.

**Tags:** [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Python](https://yomu.fyi/topic/python), [Search](https://yomu.fyi/topic/search)

[Read original post](https://huggingface.co/blog/image-search-datasets)