Unlock the power of images with AI Sheets

Summary

Hugging Face released an update to AI Sheets, an open-source tool for transforming and enriching datasets without code. The update adds vision support powered by Inference Providers, enabling users to upload images, extract structured data, generate visuals, and perform image edits in a spreadsheet interface. Each column can execute AI actions configured with custom prompts and selected models. For text extraction and reasoning tasks, AI Sheets supports vision language models including Qwen/Qwen2.5-VL-7B-Instruct and Qwen/Qwen3-VL-235B-A22B-Reasoning, while image transformations use models like Qwen-Image-Edit. Users can iterate on prompts, provide feedback using thumbs-up ratings to create few-shot examples, and export completed datasets to the Hugging Face Hub, CSV, or Parquet.

Context

Images such as receipts, product photos, charts, and scanned documents contain structured information that previously required manual effort or custom scripting to extract, analyze, or generate.

Approach / What changed

Hugging Face added vision support to AI Sheets via Inference Providers, allowing users to upload image datasets and configure column-level AI actions for transcription, classification, image-to-image editing, and text-to-image generation directly within a spreadsheet environment.

Takeaways

• AI Sheets allows column-level AI actions to extract structured text, categorize images, generate visuals from text, or edit existing images in a spreadsheet interface.
• User feedback such as manual edits and thumbs-up ratings acts as few-shot examples to improve subsequent model generation results across a column.
• Completed datasets enriched with visual or text transformations can be exported directly to CSV, Parquet, or the Hugging Face Hub as public or private repositories.

Related reading

• huggingface.co · Apr 28, 2022

  Opinion Classification with Kili and HuggingFace AutoTrain

  Understanding mobile application user feedback often requires expensive and labor-intensive manual analysis. To streamline this process, an active learning pipeline was constructed using Kili for data annotation alongside HuggingFace AutoTrain and standard transformers for text classification. The workflow utilized approximately 40,000 Medium reviews collected from the Google Play Store, classifying content into categories such as Subscription, Content, Interface, and User Experience. Hyperparameter searches with Ray backend were compared against AutoTrain, which automated training within 30 minutes. Subsequent sentiment analysis of the categorized reviews revealed that while overall sentiment remained positive, subscription and interface categories contained predominantly negative feedback, especially in application version 4.5.

  Alper

• huggingface.co · Jun 23, 2022

  Getting Started With Embeddings

  Embeddings represent unstructured information such as text and images as numerical vectors in a shared semantic space. To demonstrate their utility, a simple semantic search engine is built over US Social Security Medicare frequently asked questions. The system generates 384-dimensional vector representations for thirteen FAQ entries by dispatching POST requests to the Hugging Face Inference API using the sentence-transformers/all-MiniLM-L6-v2 model. Incoming user queries are converted into matching vector representations and evaluated against stored dataset vectors using the util.semantic_search function from the Sentence Transformers library. By calculating cosine similarity scores, the system retrieves and ranks the five most semantically relevant questions without requiring custom keyword rules or massive labeled training sets.