---
title: "Open Source"
description: "568 posts about Open Source, summarised, each linking to the original."
---

# Open Source
> 568 posts about Open Source, summarised, each linking to the original.

## Articles

### [Supercharge your OCR Pipelines with Open Models](https://yomu.fyi/post/supercharge-your-ocr-pipelines-with-open-models.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: merve, Aritra Roy Gosthipaty, Daniel van Strien, Hynek Kydlicek, Andres Marafioti, Vaibhav Srivastav, Pedro Cuenca
- Published: Oct 21, 2025
- Type: Explainer

Modern vision-language OCR models can transcribe handwritten text, multiple scripts, mathematical expressions, and chemical formulas while also interpreting tables, charts, and images. The guide compares open OCR models and explains how output choices—including DocTags, HTML, Markdown, and structured JSON—affect digital reconstruction, LLM input, and programmatic processing. It also covers locality awareness through anchors or bounding boxes, prompt-based task switching, fine-tuning versus out-of-the-box use, and factors for selecting a model. Practical deployment options include local or managed inference, vLLM, Hugging Face Endpoints, and batch scripts that add OCR results to datasets. The discussion extends document AI beyond transcription to visual retrieval and direct document question answering with vision-language models, especially where converted text can lose layout or visual context.


### [Unlock the power of images with AI Sheets](https://yomu.fyi/post/unlock-the-power-of-images-with-ai-sheets.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Ame Vi, Daniel Vila, Francisco Aranda, Damián Pumar, Leandro von Werra, Thomas Wolf
- Published: Oct 21, 2025
- Type: Announcement

Hugging Face released an update to AI Sheets, an open-source tool for transforming and enriching datasets without code. The update adds vision support powered by Inference Providers, enabling users to upload images, extract structured data, generate visuals, and perform image edits in a spreadsheet interface. Each column can execute AI actions configured with custom prompts and selected models. For text extraction and reasoning tasks, AI Sheets supports vision language models including Qwen/Qwen2.5-VL-7B-Instruct and Qwen/Qwen3-VL-235B-A22B-Reasoning, while image transformations use models like Qwen-Image-Edit. Users can iterate on prompts, provide feedback using thumbs-up ratings to create few-shot examples, and export completed datasets to the Hugging Face Hub, CSV, or Parquet.


### [AI for Food Allergies](https://yomu.fyi/post/ai-for-food-allergies.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Ludovico Comito, Antonis Vozikis, Vaibhav Pandey, Kisejjere Rashid
- Published: Oct 16, 2025
- Type: Announcement

Global food allergies affect an estimated 220 million individuals, yet computational progress in biomedical discovery remains constrained by fragmented and inaccessible scientific data. The AI for Food Allergies initiative addresses this challenge by establishing an open research community and releasing the curated Awesome Food Allergy Datasets collection across multiple biological layers. Computational pipelines leverage deep learning models, such as AllergenAI and NetAllergen-1.0, which incorporate sequence motifs and computationally predicted MHC class II presentation propensities to evaluate allergenicity. Additionally, molecular property prediction benchmarks like QM9 provide high-accuracy quantum-mechanical properties for approximately 134,000 molecules, supporting generative and virtual screening workflows targeting IgE–FcεRI binding. These combined efforts systematically structure molecular, clinical, and chemogenomic resources to accelerate allergy diagnostics and therapeutic protein engineering.


### [Get your VLM running in 3 simple steps on Intel CPUs](https://yomu.fyi/post/get-your-vlm-running-in-3-simple-steps-on-intel-cpus.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Ezequiel Lanza, Helena, Nikita, Ella Charlaix, Ilyas Moutawwakil
- Published: Oct 15, 2025
- Type: Tutorial

Deploying Vision Language Models locally presents computational challenges despite providing data privacy and independence from external servers. Optimum Intel and OpenVINO address this by optimizing lightweight models like SmolVLM2-256M for low-resource hardware, including Intel CPUs and GPUs. The process involves exporting the model to the OpenVINO Intermediate Representation format, applying 8-bit weight-only or static quantization, and executing inference with the transformed model. In single-image CPU benchmarks on an Intel Core Ultra 7 265K processor, OpenVINO with 8-bit weight-only quantization reduced time to first token from 5.150 seconds under PyTorch down to 0.247 seconds. The optimized configuration also raised decoding throughput from 0.722 to 63.928 tokens per second.


### [Nemotron-Personas-India: Synthesized Data for Sovereign AI](https://yomu.fyi/post/nemotron-personas-india-synthesized-data-for-sovereign-ai.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Kiran Praveen, Utkarsh Vaidya, Evan A, Lipika Ramaswamy, Dhruv Nathawani, Dane Corneil, Yev Meyer
- Published: Oct 13, 2025
- Type: Announcement

Most open artificial intelligence datasets reflect Western norms and English-only contexts, limiting model effectiveness across India's multilingual and multi-script landscape. To address this shortage of culturally grounded training data, NVIDIA released Nemotron-Personas-India under a CC BY 4.0 license. The synthetic dataset contains 21 million personas across 3 million records spanning 7.7 billion tokens in English, Devanagari Hindi, and Latin-script Hindi. The generation pipeline utilized NeMo Data Designer, an Apache-2.0 licensed Probabilistic Graphical Model for demographic grounding against 2011 Census distributions, and GPT-OSS-120B for narrative text. All records are entirely synthetic to eliminate re-identification risks while enabling fine-tuning for region-aware AI agents and domain-specific copilots.


### [BigCodeArena: Judging code generations end to end with code executions](https://yomu.fyi/post/bigcodearena-judging-code-generations-end-to-end-with-code-executions.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Terry Yue Zhuo
- Published: Oct 7, 2025
- Type: Announcement

Assessing AI-generated code via static inspection is error-prone and fails to reflect whether code functions correctly across real-world environments. BigCodeArena addresses this evaluation bottleneck as a human-in-the-loop platform that executes model outputs in real time inside isolated sandbox environments across ten programming languages and eight execution frameworks. Users interact directly with live applications, refine implementations through multi-turn conversations, and submit pairwise preference votes to generate Elo ratings via the Bradley-Terry model. In five months of community testing covering over 14,000 conversations, proprietary models including o3-mini and o1-mini led overall Elo rankings, while open models like Qwen2.5 and Llama-3.3-70B trailed closed competitors. The project has open-sourced its full evaluation pipelines, conversation datasets, and related benchmarks to support community-driven code generation assessment.


### [VibeGame: Exploring Vibe Coding Games](https://yomu.fyi/post/vibegame-exploring-vibe-coding-games.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Dylan Ebert
- Published: Sep 29, 2025
- Type: Problem & solution

Vibe coding games often fails as projects grow because large context sizes degrade model performance. After evaluating Roblox MCP, Unity MCP, and open web libraries with Claude Code, the author found web tooling offered the strongest model proficiency but lacked high-level game abstractions. To bridge this gap, the author created VibeGame, an open-source declarative game engine built on three.js, rapier, and bitecs. VibeGame uses an Entity-Component-System architecture, XML-like markup, and bundled documentation in an llms.txt file to keep project context lean. While effective for basic physics and rendering mechanics, the engine requires further development for complex features like inventories, multiplayer, and interactions.


### [Swift Transformers Reaches 1.0 – and Looks to the Future](https://yomu.fyi/post/swift-transformers-reaches-1-0-and-looks-to-the-future.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Pedro Cuenca, Christopher Fleetwood, Mattt, Vaibhav Srivastav
- Published: Sep 26, 2025
- Type: Announcement

swift-transformers has released version 1.0 to provide a stable foundation for running local models on Apple Silicon platforms. The library supplies components missing from Core ML and MLX, focusing heavily on tokenization, chat templates, and Hugging Face Hub integration. In this major release, Tokenizers and Hub become standalone top-level modules, enabling developers to import only the components they require. The update also integrates an updated swift-jinja library for faster template processing, adopts Modern Core ML stateful models and MLTensor APIs, removes unnecessary CLI dependencies, and adds full Swift 6 support. Future development will prioritize deeper integration with MLX frameworks and agentic workflows like MCP.


### [Smol2Operator: Post-Training GUI Agents for Computer Use](https://yomu.fyi/post/smol2operator-post-training-gui-agents-for-computer-use.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Amir Mahla, merve, Sergio Paniego, Vaibhav Srivastav, Lewis Tunstall
- Published: Sep 23, 2025
- Type: Problem & solution

Lightweight vision-language models often lack the precise visual grounding and action execution capabilities required for graphical user interface automation. To resolve this deficiency, the authors constructed a post-training pipeline that evolves base models into agentic GUI coders through supervised fine-tuning. The process standardizes disparate mobile and desktop dataset action spaces into a unified functional format utilizing normalized coordinates relative to image size. Training is structured into two sequential phases that first establish basic visual perception and subsequently introduce explicit reasoning steps with masked assistant loss. On the ScreenSpot-v2 perception benchmark, fine-tuning SmolVLM2-2.2B-Instruct increased element localization accuracy to 61.71 percent, while a compact 460-million-parameter model achieved approximately 58 percent accuracy.


### [Simplify AI-Driven Data Connectivity With MongoDB and MCP Toolbox](https://yomu.fyi/post/simplify-ai-driven-data-connectivity-with-mongodb-and-mcp-toolbox.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Venkatesh Shanbhag, Yang Li, Kurtis Van Gent
- Published: Sep 22, 2025
- Type: Announcement

Organizations building generative AI applications require simplified, standardized connections to an expanding ecosystem of enterprise databases. MCP Toolbox for Databases, an open-source Model Context Protocol server based on Anthropic's standardized protocol, now supports MongoDB Atlas to streamline connections to generative AI systems. This new integration allows gen AI agents to execute standard CRUD operations alongside advanced transformations and computations using the MongoDB aggregation framework. Through flexible configuration files, developers can configure MongoDB Atlas as both a data source and a sink by defining parameters like connection URIs, target datasets, filters, projections, and toolsets. By linking flexible document data with cross-database query capabilities, enterprises can unify results and automate multi-source AI workflows.


### [SyGra: The One-Stop Framework for Building Data for LLMs and SLMs](https://yomu.fyi/post/sygra-the-one-stop-framework-for-building-data-for-llms-and-slms.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Bidyapati Pradhan, Vipul Mittal, Amit Kumar Saha, Surajit Dasgupta
- Published: Sep 22, 2025
- Type: Problem & solution

Training and aligning large language models and small language models requires datasets tailored for complex reasoning, preference tuning, multi-turn questioning, and format conversion. To eliminate the need for bespoke data preparation scripts, SyGra provides a low-code and no-code Python framework for automated dataset creation, transformation, and alignment. The library integrates with diverse inference backends, including vLLM, Hugging Face TGI, Triton, and Ollama, allowing teams to focus on prompt engineering rather than pipeline infrastructure. It supports data workflows ranging from converting PDFs and knowledge bases into question-answering pairs to filtering low-quality samples and generating Direct Preference Optimization datasets. These plug-and-play workflows reduce manual curation effort while accelerating model fine-tuning and retrieval-augmented generation pipelines.


### [Gaia2 and ARE: Empowering the community to study agents](https://yomu.fyi/post/gaia2-and-are-empowering-the-community-to-study-agents.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Clémentine Fourrier, Grégoire Mialon, Maxime Lecanu, Pierre Andrews, Adrien Carreira, frere thibaud, Avijit Ghosh, Romain Froger, Dheeraj Mekala, Caroline Pascal, Ulyana Piterbarg
- Published: Sep 22, 2025
- Type: Benchmark

Existing AI agent evaluation environments are often tightly coupled to specific tasks and fail to model real-world challenges such as API failures, spontaneous events, and asynchronous conditions. To address this limitation, the Gaia2 benchmark and the Meta Agents Research Environments (ARE) framework introduce interactive read-and-write evaluation. Gaia2 incorporates 1,000 human-created scenarios spanning multi-step execution, cross-source search, ambiguity handling, adaptability, temporal reasoning, agent collaboration, and noise tolerance. Using a simulated smartphone interface equipped with 101 tools, evaluations showed that GPT-5 with high reasoning scored highest overall, while Kimi K2 was the leading open-source model. The results demonstrated that instruction following and search do not reliably predict performance on closer-to-real-world tasks.


### [Democratizing AI Safety with RiskRubric.ai](https://yomu.fyi/post/democratizing-ai-safety-with-riskrubric-ai.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Gal Moyal
- Published: Sep 18, 2025
- Type: Announcement

Cloud Security Alliance and Noma Security introduced RiskRubric.ai to provide standardized, transparent risk assessments across the open AI model ecosystem. The framework evaluates AI models across six pillars—transparency, reliability, security, privacy, safety, and reputation—using over 1,000 reliability tests, 200 adversarial security probes, automated code scanning, and harmful content evaluations. Each model receives 0–100 scores and A–F letter grades, supplemented by specific vulnerability findings and recommended mitigation strategies to assist deployment filtering. Initial benchmark results across models showed composite scores ranging from 47 to 94 with a median of 81, revealing polarized safety distributions and indicating that security hardening directly correlates with reduced safety risks.


### [Public AI on Hugging Face Inference Providers 🔥](https://yomu.fyi/post/public-ai-on-hugging-face-inference-providers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Joseph Low, Joshua Tan, Célina Hanouti, Julien Chaumond, Simon Brandeis, Lucain Pouget
- Published: Sep 17, 2025
- Type: Announcement

Hugging Face has integrated Public AI as a supported Inference Provider on the Hugging Face Hub. Public AI operates as a nonprofit, open-source project providing access to sovereign and public models from institutions such as the Swiss AI Initiative and AI Singapore. Its distributed infrastructure combines a vLLM-powered backend serving OpenAI-compatible APIs across partner-donated clusters with a global load-balancing routing layer. Users can access these models through the Hugging Face web UI, Python client SDK, and JavaScript SDK using either direct provider API keys or routed Hugging Face tokens. At the time of announcement, inference through the Public AI provider is free of charge, supported by donated GPU time and advertising subsidies.


### [How we built it: Real-time analytics for Stripe Billing](https://yomu.fyi/post/how-we-built-it-real-time-analytics-for-stripe-billing.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Reed Trevelyan
- Published: Sep 16, 2025
- Type: Problem & solution

Stripe built a real-time streaming analytics system for Billing after customers said they needed to detect emerging behavior and adapt pricing quickly, replacing a batch pipeline whose subscription updates lagged by 24 hours on average. The architecture uses Apache Flink to maintain compressed subscription history and incrementally process analytics events, while a custom Apache Spark job generates initial Flink state from billions of historical events and supports validation and exports. A new Apache Pinot v2 query engine performs windowed aggregations and joins without offline preaggregation; most updates are processed in under one minute, nearly all reach users within 15 minutes, and production query latency is under 300 milliseconds. Metric-definition changes trigger historical recalculation while new events are buffered and later replayed, preserving a consistent Dashboard view during the transition.


### [Visible Watermarking with Gradio](https://yomu.fyi/post/visible-watermarking-with-gradio.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Margaret Mitchell
- Published: Sep 15, 2025
- Type: Announcement

Generative artificial intelligence tools produce images, video, audio, and text that have become increasingly difficult to distinguish from real-world media captures. To improve synthetic media transparency, Hugging Face introduced visible watermarking capabilities directly into the Gradio web application framework. Developers can now overlay visual watermarks on image and video outputs by specifying a single watermark parameter using file paths, open image objects, or NumPy arrays. The implementation also supports QR code watermarks, which can match the visual style of generated media while conveying detailed background information. Furthermore, the Gradio Chatbot component accepts a text watermark parameter that automatically attaches attribution metadata whenever users copy generated responses to their clipboards.


### [Introducing the Palmyra-mini family: Powerful, lightweight, and ready to reason!](https://yomu.fyi/post/introducing-the-palmyra-mini-family-powerful-lightweight-and-ready-to.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Rakshith, Tom Peres
- Published: Sep 11, 2025
- Type: Announcement

WRITER released three open models in the Palmyra-mini family ranging between 1.5B and 1.7B parameters, designed for efficient inference and specialized reasoning tasks. Built on the Qwen architecture, the release includes the standard palmyra-mini base model alongside two Chain of Thought variants, palmyra-mini-thinking-a and palmyra-mini-thinking-b. In benchmark evaluations, palmyra-mini achieved 52.6% on Big Bench Hard, while palmyra-mini-thinking-a reached 82.87% on GSM8K and palmyra-mini-thinking-b reached 92.5% on AMC23. For palmyra-mini-thinking-b, applying reinforcement learning fine-tuning to an OpenReasoning-Nemotron-1.5B base improved single-shot pass@1 accuracy while reducing sampling diversity and majority@64 performance. The models are available in GGUF and MLX-BF16 quantizations and support inference engines including vLLM, SGLang, TRTLLM, and TGI.


### [Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers](https://yomu.fyi/post/tricks-from-openai-gpt-oss-you-can-use-with-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Aritra Roy Gosthipaty, Sergio Paniego, Vaibhav Srivastav, Pedro Cuenca, Arthur Zucker, Nathan Habib, Cyril Vallez
- Published: Sep 11, 2025
- Type: Announcement

To support OpenAI's GPT-OSS series of models, the transformers library introduced several performance upgrades that apply across supported architectures. The release integrates zero-build custom kernels downloadable directly from the Hub, reducing external dependency bloat and compilation friction for operations like Liger RMSNorm and MegaBlocks MoE. Native support for MXFP4 quantization groups vector elements into 32-value blocks with shared scales, allowing GPT-OSS 20B to fit in roughly 16 GB of VRAM and GPT-OSS 120B in roughly 80 GB. In addition, transformers incorporates Flash Attention 3 with attention sinks, continuous batching via the generate\_batch API for experimentation, and automatic memory pre-allocation to speed up model loading on GPUs.


### [Fine-tune Any LLM from the Hugging Face Hub with Together AI](https://yomu.fyi/post/fine-tune-any-llm-from-the-hugging-face-hub-with-together-ai.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Zain Hasan, Artem Chumachenko, Egor Timofeev, Max Ryabinin
- Published: Sep 10, 2025
- Type: Announcement

Together AI and Hugging Face have introduced an integration enabling developers to fine-tune compatible Hugging Face Hub models directly on Together AI's managed infrastructure. To launch a fine-tuning job via the Python SDK, users provide a base model from Together's catalog as a configuration template alongside the target Hugging Face repository identifier. This base model template dictates GPU allocation, memory configuration, training pipelines, and inference setup for custom models with matching architectures and sizes. The workflow operates bidirectionally, pulling from public or token-authenticated private repositories and optionally pushing completed checkpoints back to the Hub upon completion. Geared toward CausalLM models under 100 billion parameters, the capability enables faster iteration cycles and domain adaptation without custom DevOps infrastructure.


### [Jupyter Agents: training LLMs to reason with notebooks](https://yomu.fyi/post/jupyter-agents-training-llms-to-reason-with-notebooks.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Baptiste Colle, Hanna Yukhymenko, Leandro von Werra
- Published: Sep 10, 2025
- Type: Problem & solution

Small language models often struggle to compete with large frontier models on complex, agentic data science tasks. To improve notebook-based reasoning, researchers simplified agent scaffolding down to roughly two hundred lines of code with dedicated execution and final answer tools, boosting baseline easy accuracy on the DABStep benchmark from 44.4 percent to 59.7 percent. They constructed a curated training dataset by deduplicating two terabytes of Kaggle notebooks, automatically fetching five terabytes of linked datasets, and scoring educational value and relevance using Qwen3-32B. Question-answer pairs grounded in verified execution traces were generated to fine-tune compact Qwen3-4B thinking and instruct models. The team released the trained models alongside the dataset and execution sandboxes, establishing a foundation for reinforcement learning and distillation on notebook workflows.


[Newer posts](https://yomu.fyi/topic/open-source/page/16.md) · [Older posts](https://yomu.fyi/topic/open-source/page/18.md)