Loading…
Hugging Face
AI platform and community providing machine learning models, datasets, libraries, and tools for building AI applications.
- 318 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
Introducing NVIDIA Nemotron 3 Nano Omni: Long-Context Multimodal Intelligence for Documents, Audio and Video Agents
NVIDIA Nemotron 3 Nano Omni is introduced as an omni-modal understanding model for document analysis, multiple-image reasoning, automatic speech recognition, long audio-video understanding, agentic computer use, and general reasoning. It extends the Nemotron multimodal line from vision-language processing to text, image, video, and audio, targeting long, dense inputs such as 100-plus-page documents and mixed audio-visual evidence. Its unified encoder-projector-decoder combines a Nemotron 3 Nano 30B-A3B hybrid Mamba-Transformer MoE backbone with C-RADIOv4-H and Parakeet-TDT-0.6B-v2 encoders; dynamic visual resolution, Conv3D tubelets, and Efficient Video Sampling support multimodal context compression. The post reports leading or top results across document, video, audio-video, voice, and ASR benchmarks, alongside up to 9x higher throughput and 2.9x single-stream reasoning speed on multimodal use cases, and provides BF16, FP8, and NVFP4 checkpoints.
How to build scalable web apps with OpenAI's Privacy Filter
The article presents three web applications built around OpenAI’s open-source Privacy Filter, a PII detector that labels eight categories in one forward pass over a 128k-token context. Document Privacy Explorer extracts PDF or DOCX text and returns category-labeled spans, while Image Anonymizer combines Tesseract word boxes with character offsets to produce editable pixel redaction rectangles. SmartRedact Paste replaces detected spans with category placeholders, stores public and token-gated views, and evicts expired pastes in a daemon thread. Across the apps, gradio.Server’s @server.api routes model work through Gradio’s queue and gradio_client, while plain FastAPI routes serve custom HTML, views, and lightweight lookups. The implementations keep browser-side filtering, canvas edits, and PNG export out of server round-trips, and the paste service is about 200 lines including storage.
DeepSeek-V4: a million-token context that agents can actually use
DeepSeek V4 is presented as an open model family aimed at long-running agentic workloads, with Pro and Flash MoE checkpoints offering 1M-token context windows. Its central architecture alternates Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA), compressing KV entries by 4x and 128x respectively, while using FP8/BF16 storage and FP4 indexing. The post-training design preserves reasoning across user turns when tool calls are present, introduces a dedicated XML tool-call schema, and relies on DSec, a Rust sandbox platform for RL rollouts. At 1M tokens, Pro uses 27% of V3.2's single-token FLOPs and 10% of its KV cache, while Flash uses 10% and 7%; V4-Pro-Max reports 80.6% on SWE Verified and 0.59 MRCR 8-needle retrieval at 1M.
How to Use Transformers.js in a Chrome Extension
The guide explains how to build a Chrome extension that runs local AI with Transformers.js under Manifest V3, using a browser assistant as the reference architecture. It assigns model orchestration, lifecycle management, tool execution, and conversation state to a background service worker, while a side panel handles chat and a content script handles page extraction and highlighting. The implementation uses Gemma 4 E2B in q4f16 for text generation and all-MiniLM-L6-v2 in fp32 for embeddings, with WebGPU, DynamicCache-based KV caching, explicit download progress, and reusable pipelines. Typed message contracts, model-output parsing, and separate internal and UI transcripts support a deterministic tool loop, while chrome.storage.local, IndexedDB, and background caches place data according to lifecycle and access needs; the guide concludes that separating these runtimes keeps the UI responsive and avoids duplicate model loads.
QIMMA قِمّة ⛰: A Quality-First Arabic LLM Leaderboard
QIMMA is an Arabic LLM leaderboard built to address fragmented evaluation, translation artifacts, weak benchmark quality checks, limited coverage, and reproducibility gaps. It treats benchmark validation as a prerequisite for reported scores rather than an afterthought. The suite consolidates 109 subsets from 14 source benchmarks into more than 52,000 samples across seven domains, with 99% native Arabic content, Arabic-adapted HumanEval+ and MBPP+ coding tasks, and six prompt templates. Before model evaluation, Qwen3-235B-A22B-Instruct and DeepSeek-V3-671B assess samples using a 10-point rubric, flagged cases receive native-speaker review, and code statements are refined without changing task identifiers, reference solutions, or tests. In the April 2026 rankings, Qwen/Qwen3.5-397B-A17B-FP8 ranks first, while Arabic-specialized models lead cultural and linguistic tasks and multilingual models lead coding.
AI and the Future of Cybersecurity: Why Openness Matters
Mythos is presented as a frontier AI system that rapidly finds and patches software vulnerabilities, with the post arguing that its surrounding system matters more than the language model alone. Its recipe combines substantial compute, software-relevant training data, vulnerability-probing and patching scaffolding, speed, and some autonomy; together, these elements can uncover vulnerabilities, find exploits, and build patches. The post argues that open code and tooling can distribute detection, verification, coordination, and patch propagation across communities, while reducing capability asymmetry between attackers and defenders. For defensive use, it recommends semi-autonomous agents with prespecified actions, human approval, auditable logs, and private deployment using open components and existing security tools. This approach is especially relevant to high-stakes organizations that need inspectable, controllable systems and protection for sensitive data.
Ecom-RLVE: Adaptive Verifiable Environments for E-Commerce Conversational Agents
Ecom-RLVE extends the RLVE framework from single-turn reasoning puzzles to multi-turn, tool-augmented e-commerce conversations. EcomRLVE-GYM supplies eight environments spanning product discovery, substitution, cart building, returns, order tracking, policy QA, bundle planning, and multi-intent journeys, with procedural problem generation, a 12-axis difficulty curriculum, and algorithmically verifiable rewards. Each episode requires tool use and world-state changes; rewards combine task completion, efficiency, and a penalty for recommending product IDs not retrieved, while malformed outputs and illegal calls fail immediately. Adaptive scheduling advances each environment independently when its success rate is reliable. An initial study trained Qwen 3 8B with DAPO for 300 steps on cart building and reported progressively increasing difficulty reached, consistent with a steady learning signal.
The PR you would have opened yourself
mlx-lm contributors can use a Skill and a separate test harness to port language models from transformers, with the goal of making new architectures available shortly after their upstream implementations land. The work addresses rising agent-generated pull-request volume and the difficulty of preserving implicit codebase conventions, user contracts, and performance-sensitive behavior that agents may not understand. Given a conversion prompt, the Skill sets up environments, finds model variants and checkpoints, reads transformers modeling code, writes an MLX implementation, and iterates through tests including dtype checks, generation examples, numerical comparisons, and per-layer comparisons. The resulting pull requests disclose agent assistance and include detailed verification data, while contributors remain responsible for reviewing and accepting the results. Current limitations include shared-utility refactors, vision-language models, quantized uploads, and thinking-specific tests.
Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers
Sentence Transformers provides a Python library for training and using embedding and reranker models across text, images, audio, and video in retrieval and semantic search applications. The post shows how to finetune multimodal models on task-specific data, using Visual Document Retrieval as the example: matching text queries with document page images while preserving charts, tables, and layout. It describes the SentenceTransformerTrainer pipeline, model and processor configuration, dataset handling, and alternative Router-based encoders, while also covering multimodal reranker training with BinaryCrossEntropyLoss and direction-specific datasets. On its evaluation data, finetuning Qwen/Qwen3-VL-Embedding-2B raised NDCG@10 from 0.888 to 0.947 and outperformed all tested VDR models, including models up to 4x its size.
Inside VAKRA: Reasoning, Tool Use, and Failure Modes of Agents
VAKRA is an executable, tool-grounded benchmark for evaluating whether AI agents can complete enterprise-like workflows that combine API calls, document retrieval, and natural-language constraints. It provides more than 8,000 locally hosted APIs backed by real databases across 62 domains, plus domain-aligned documents, and uses full execution traces to assess multi-step behavior. The four capability groups cover API chaining, dashboard-API tool selection, multi-hop dashboard reasoning, and multi-hop multi-source reasoning with multi-turn dialogue and optional tool-use policies; tasks can require three to seven reasoning steps, while some segments specify one to twelve tool calls or up to five logical hops. Reported analyses find that models perform best on single-hop questions, degrade as API hops increase, and often violate policies or fail to retrieve enough information when constraints restrict the relevant source, exposing a gap between isolated tool competence and end-to-end reliability.
Meet HoloTab by HCompany. Your AI browser companion.
HoloTab is a Chrome extension that brings HCompany’s computer-use AI into the browser. It navigates websites, fills fields, and makes decisions from a user’s description, with vision models, action planning, and interface understanding operating underneath without setup or technical skills. The announcement follows the March 31st release of Holo3, described as the company’s most advanced computer-use model to date. For repetitive workflows, users can record their actions and narration in a tab; HoloTab turns the recording into a routine that can later be rerun or scheduled. The stated goal is to make computer-use AI accessible beyond teams with engineers who can connect such systems, and the extension is presented as free and available for everyone today.
Waypoint-1.5: Higher-Fidelity Interactive Worlds for Everyday GPUs
Overworld’s Waypoint-1.5 is a real-time video world model aimed at making interactive generative worlds usable on consumer hardware rather than limited to datacenter-scale compute. It offers two tiers: a 720p model for higher-performance desktop GPUs, capable of up to 60 FPS on RTX 3090 through 5090 hardware, and a 360p model for broader deployment, including gaming laptops and soon Apple Silicon Macs. Training used nearly 100 times more data than Waypoint-1, while more efficient video-modeling techniques reduce redundant computation across frames to support coherent motion and responsive exploration. Users can run it locally through the Biome desktop client, try it in a browser via Overworld Stream, or use the World Engine inference library and third-party clients. The release presents improved visual fidelity, broader local hardware support, and continued progress toward real-time interactive world simulation.
Multimodal Embedding & Reranker Models with Sentence Transformers
Sentence Transformers v5.4 adds multimodal embedding and reranking support for text, images, audio, and video through its Python API. Embedding models map modalities into a shared space, while multimodal Cross Encoder rerankers score relevance for mixed-modality pairs, enabling cross-modal similarity, retrieval, and multimodal RAG workflows. The guide shows model loading, URL-based image encoding, encode_query() and encode_document() prompt handling, similarity calculation, and ranking, with a Qwen3-VL-Embedding-2B example producing 2,048-dimensional image vectors and matching text queries to images. It notes that cross-modal scores can be lower because of the modality gap, while relative ordering can still support retrieval, and that rerankers generally offer higher quality at the cost of slower per-pair processing; installation extras and GPU memory requirements vary by modality and model.
Safetensors is Joining the PyTorch Foundation
Safetensors has joined the PyTorch Foundation as a foundation-hosted project under the Linux Foundation, alongside DeepSpeed, Helion, Ray, vLLM, and PyTorch. It began as a Hugging Face project addressing the risk of arbitrary code execution in pickle-based model-weight formats. Its format stores tensor metadata in a JSON header capped at 100MB, followed by raw tensor data, and supports zero-copy mapping and lazy loading. The move places the trademark, repository, and governance with the Linux Foundation, while Luc and Daniel remain on the Technical Steering Committee and lead day-to-day work. Existing users get the same format, APIs, and Hub integration without breaking changes, while contributors gain a documented maintainer path; planned work includes PyTorch core serialization, accelerator-aware loading, parallel loading, and quantization support.
Welcome Gemma 4: Frontier multimodal intelligence on device
Google DeepMind’s Gemma 4 family is available on Hugging Face as Apache 2-licensed, multimodal models intended for use across agents, inference engines, fine-tuning libraries, and on-device deployments. The family accepts image and text inputs across five sizes, while E2B, E4B, and 12B Unified also support audio; context windows range from 128k to 256k tokens. Its design combines alternating local and global attention, dual RoPE configurations, Per-Layer Embeddings, shared KV caches, and model-specific vision and audio handling, including an encoder-free unified 12B variant. The post documents integrations for Transformers, llama.cpp, MLX, WebGPU, Rust, and other tools, alongside fine-tuning examples and demos. Reported instruction-tuned results include a 1452 estimated text-only LMArena score for 31B and benchmark results spanning reasoning, coding, vision, audio, and long-context tasks.
Falcon Perception
Falcon Perception is a 0.6B-parameter early-fusion Transformer for open-vocabulary grounding and segmentation from natural-language prompts. It processes image patches, text, and task tokens in one sequence, using hybrid attention so image tokens receive bidirectional context while text and task tokens decode causally. Its Chain-of-Perception interface predicts each instance as coordinate, size, and segmentation tokens, with specialized Fourier-feature and dot-product heads for dense outputs. The post also introduces PBench, which profiles attributes, OCR-guided identification, spatial understanding, relations, and crowded scenes. Falcon Perception reaches 68.0 Macro-F1 on SA-Co versus 62.3 for SAM 3, while presence calibration remains weaker at MCC 0.64 versus 0.82; the 0.3B Falcon OCR model scores 80.3 on olmOCR and 88.6 on OmniDocBench.
Any Custom Frontend with Gradio's Backend
gradio.Server enables applications built with React, Svelte, or plain HTML/JavaScript to use Gradio’s backend facilities instead of Gradio components. The post demonstrates this with Text Behind Image, whose browser UI layers a background, styled text, and a transparent foreground produced by BiRefNet, then exports a PNG client-side. A roughly 50-line Python backend extends FastAPI, exposes a standard HTML route, and defines an @app.api("remove_background") endpoint; @spaces.GPU handles ZeroGPU allocation while Gradio supplies queuing, concurrency control, SSE streaming, and gradio_client compatibility. The frontend calls the endpoint through the Gradio JavaScript client, so image-processing requests pass through the queue while rendering and compositing remain in the browser. The result is a custom frontend that retains Spaces hosting and Gradio infrastructure.
Granite 4.0 3B Vision: Compact Multimodal Intelligence for Enterprise Documents
Granite 4.0 3B Vision is a compact vision-language model designed to extract information from enterprise documents, including tables, charts, forms, and semantically meaningful key-value pairs. It combines a million-scale ChartNet dataset with a code-guided synthesis pipeline, DeepStack Injection for routing semantic and high-resolution visual features to different layers, and a LoRA adapter architecture built on Granite 4.0 Micro. The modular design supports standalone image processing, text-only fallback, and integration with Docling for multi-page document pipelines. Reported evaluations show 86.4% on Chart2Summary, leading the evaluated models, second place on Chart2CSV at 62.1%, leading scores on the listed table benchmarks, and 85.5% zero-shot exact-match accuracy on VAREX.
Training mRNA Language Models Across 25 Species for $165
OpenMed describes an end-to-end protein engineering pipeline that connects ESMFold structure prediction, ProteinMPNN sequence design, and custom mRNA codon optimization. The codon-language-model study compares CodonBERT, ModernBERT, and several CodonRoBERTa variants using 250,000 E. coli coding sequences, a 69-token codon vocabulary, masked-language modeling, and training on four 80GB A100 GPUs. CodonRoBERTa-large-v2 achieved perplexity 4.10, CAI Spearman correlation 0.404, and 7.7% synonymous recovery, while ModernBERT-base reached perplexity 26.24 and correlation 0.070. The work then trained a multi-species suite on 381,283 sequences spanning 25 species, producing four production models in 55 GPU-hours and documenting planned extensions such as larger training data and additional prediction heads.
TRL v1.0: Post-Training Library Built to Move with the Field
TRL v1.0 marks a transition from research codebase to a library intended for production systems, with clearer stability expectations. It implements more than 75 post-training methods, but its design prioritizes trying, comparing, and using them over coverage alone. The library responds to a field in which PPO, preference optimization methods such as DPO, ORPO, and KTO, and RLVR-style approaches such as GRPO repeatedly changed which components of the training stack appear essential. TRL addresses that volatility with a stable core under semantic versioning, an experimental surface for faster-moving methods, and deliberately limited abstractions that favor explicit implementations over generic hierarchies. v1.0 also follows distributed breaking changes across the 0.x releases, so migration from the last 0.x version is described as minimal, while future work targets asynchronous GRPO, scaling, and actionable training diagnostics.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.