Loading…
Hugging Face
AI platform and community providing machine learning models, datasets, libraries, and tools for building AI applications.
- 318 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
Designing the hf CLI as an agent-optimized way to work with the Hub
The post presents a redesign of hf, the Hugging Face Hub’s official CLI, for both human users and coding agents that download, upload, manage repositories, run Jobs, and administer other Hub resources. The CLI detects agents through environment variables, switches to compact, complete TSV output, and supports explicit human, agent, JSON, and quiet formats; agent mode was introduced in hf v1.9.0. Next-command hints, stderr separation, non-interactive failure messages, retry-safe operations, confirmations, and dry-run previews support multi-step automation without removing human-friendly terminal behavior. Benchmarks report equal-or-better success with roughly 1.3× to 1.8× fewer tokens than curl or the SDK on comparable tasks, and up to 6× fewer on complex workflows; the generated hf-cli skill reduced tool calls by about 30% without reducing token use or improving reliability.
Direct Preference Optimization Beyond Chatbots
DharmaOCR applies Direct Preference Optimization (DPO) beyond chat alignment to reduce text degeneration in structured OCR, where outputs are either acceptable transcriptions or failure loops. In a benchmark of Brazilian Portuguese structured-document OCR, vanilla degeneration rates among open-source model families ranged from below 1% to above 33%, while supervised fine-tuning (SFT) did not consistently reach production-acceptable levels. After SFT, the pipeline generated multiple candidates for 23,726 documents, used an automated LLM judge to score them, and retained the model’s degenerate outputs as rejected examples paired with chosen outputs. DPO reduced degeneration in every tested family, by 59.4% on average and up to 87.6%, without reducing extraction quality according to the benchmark; applicability is presented as dependent on failures being categorically identifiable, reliably scoreable, and sufficiently numerous.
Adding MCP Tools to Reachy Mini
Reachy Mini’s conversation app can now call tools hosted in public Hugging Face Spaces through MCP, allowing capabilities such as weather checks and web search to be added without editing the app or downloading tool code locally. The app retains built-in robot tools and custom local tools, while profiles use tools.txt to control which tools the model may call. The tool-spaces command validates a Space, probes its MCP endpoint, discovers tool IDs, and can append them to a selected profile; remote names are namespaced to avoid collisions. The feature supports public MCP-compatible Gradio Spaces, multiple Spaces, and backend-agnostic registration, but not private or authenticated Spaces, non-Gradio Spaces, arbitrary MCP URLs, or guaranteed parallel orchestration.
Holo3.1: Fast & Local Computer Use Agents
Holo3.1 is a family of computer-use models released to improve robustness across web, desktop, and mobile environments, agent frameworks, and deployment targets. Based on Qwen, it adds native function-calling alongside structured JSON outputs, expands mobile automation, and offers 0.8B, 4B, 9B, and 35B-A3B sizes for different cost, latency, and performance needs. On AndroidWorld, the 35B-A3B model rises from 67% to 79.3%, while the 4B and 9B variants rise from 58% to 72%; function-calling and native execution achieve near-parity across the stated benchmarks, and Holo3.1 improves more than 25% over Holo3 in the Holotab harness. The release also provides 35B-A3B FP8, Q4 GGUF, and NVFP4 checkpoints for local inference: NVFP4 matches FP8 on OSWorld at about two points below BF16, reaches 1.74 times BF16 token throughput on DGX Spark, and with harness optimizations cuts average step time from 6.8 seconds to 3.3 seconds.
Introducing Mellum2: A 12B Mixture-of-Experts Model by JetBrains
JetBrains is releasing Mellum2, a 12B-parameter Mixture-of-Experts model trained from scratch on natural language and code. It targets latency-sensitive, high-throughput workloads including routing, retrieval-augmented generation (RAG), summarization, sub-agents, and coding features. The model has 12B total parameters but activates 2.5B per token, with its architecture keeping total capacity high while making inference more efficient for real-time use. Released under the Apache 2.0 license, Mellum2 is described as competitive with similarly sized open models on code generation, reasoning, science, and math benchmarks while delivering more than 2x faster inference. Its text-and-code focus supports high-throughput production workloads and private deployments where proprietary code or internal data may be involved.
Beyond LLMs: Why Scalable Enterprise AI Adoption Depends on Agent Logic
Enterprise workflows are dynamic, long-running, API- and data-rich, and constrained by policies or regulations, creating challenges that an LLM alone may not handle efficiently. The article defines agent logic as software primitives—including knowledge graphs, algorithms, and program-analysis libraries—within an agent harness that steer models toward relevant workflow context. Examples span legacy COBOL and PL/1 understanding, automated test generation, incident investigation and code remediation, and compliance modernization; reported results include ~30× lower token consumption for application understanding, 20–45% higher test coverage, and up to 4.0× better incident-investigation performance than a ReAct baseline. A Condition Insights example reports 57% fewer unsupported claims and 77% lower average token usage, supporting the conclusion that structured guidance can improve quality, cost, and rule adherence.
Profiling in PyTorch (Part 1): A Beginner's Guide to torch.profiler
The post introduces torch.profiler through a beginner-oriented walkthrough of profiling a matrix multiplication and addition, explaining how to configure the profiler and interpret its outputs. It uses a small function with torch.matmul and torch.add, annotates it with record_function, captures CPU and CUDA activities after warm-up iterations, and exports a profiler table plus a Chrome trace. The table summarizes event counts and CPU or GPU time, while the trace shows execution order, call hierarchy, CPU-GPU gaps, kernel launches, and overlap. Comparing 64x64 and 4096x4096 matrices demonstrates a shift from overhead-bound behavior to compute-bound behavior; the guide also covers dispatch chains, hidden device-to-device copies, profiler buffer activity, and torch.compile's runtime and caching behavior.
Reachy Mini goes fully local
Reachy Mini’s conversation app can now run its speech stack locally instead of sending audio to a server, avoiding cloud, API keys, and data leaving the machine. The tutorial uses speech-to-speech as a cascaded VAD → STT → LLM → TTS pipeline exposed through a Realtime API-compatible /v1/realtime WebSocket, with Silero VAD v5, Parakeet-TDT 0.6B v3, Qwen3-TTS, and a selectable LLM backend. It shows llama.cpp serving Gemma 4 E4B with parallel slots, 64k context, flash attention, and full sliding-window attention, then covers vLLM, MLX, Transformers, hosted Responses API endpoints, and network connection to a wireless robot. Once the backend and app are connected, conversations run through the local loop, while individual stages can be exchanged to trade latency, quality, language coverage, and hardware choices.
Shipping a Trillion Parameters With a Hub Bucket: Delta Weight Sync in TRL
Async RL weight synchronization currently requires the trainer to send a complete model to the inference engine after each optimizer step, creating a critical-path transfer that can reach roughly a terabyte for a frontier 1T-parameter checkpoint. The article attributes the opportunity for reduction to bf16 arithmetic: at RL learning rates, many updates fall below the bf16 visibility threshold, leaving about 99% of weights bit-identical between steps and more than 98% even in the reported worst case. TRL's delta-weight-sync implementation records changed elements around the optimizer step, encodes them in sparse safetensors, uploads each delta to a Hugging Face Bucket, and lets vLLM fetch and apply it. On Qwen3-0.6B, this reduces each per-step payload from 1.2 GB to 20–35 MB, while a demonstrated setup connected separate trainer, vLLM, and Wordle Spaces through one bucket. Open issues include extra CPU bf16 snapshots, fixed anchor cadence, and unmeasured multi-node FSDP2 support.
Harness, Scaffold, and the AI Agent Terms Worth Getting Right
AI-agent terminology has outpaced shared definitions, leaving terms such as harness and scaffold blurred across frameworks and contexts. The glossary separates the model, which produces text but has no memory or execution loop, from scaffolding, which defines instructions, tools, output parsing, and context management, and the harness, which runs model calls, routes tool use, and decides when to stop. It presents an agent as a model plus the surrounding system that lets it act repeatedly, while distinguishing tools, skills, sub-agents, and orchestrators by their roles. For training, it defines the environment, trainer, rollout, and reward, including verifiable or learned scores and sparse or dense feedback. Because usage is not universal, the glossary offers a practical mental model rather than enforcing one vocabulary.
OlmoEarth v1.1: A more efficient family of Earth observation models
OlmoEarth v1.1 is a new family of Earth observation models designed to reduce the compute required for large-scale satellite imagery predictions. Compute dominates the lifecycle from data export through preprocessing, inference, and post-processing, motivating a focus on transformer token sequence length. For Sentinel-2, the models collapse three resolution-specific tokens into one, reducing token counts threefold; this initially caused a 10 ppt drop on the m-eurosat kNN benchmark, so the pre-training regimen was modified. OlmoEarth v1.1 runs up to three times cheaper than v1 while maintaining similar performance across a mix of research benchmarks and partner-built tasks, although some regressions remain, and the same dataset is used to isolate methodological effects.
Introducing the Ettin Reranker Family
Tom Aarsen introduces six Sentence Transformers CrossEncoder rerankers, ranging from 17M to 1B parameters, alongside their training data and full recipe. Each model uses an Ettin ModernBERT encoder with unpadded attention, RoPE, GeGLU, up to 8,192 tokens of context, and a modular classification head whose ablations favored CLS pooling over mean pooling. Training applies pointwise MSE distillation to scores from mixedbread-ai/mxbai-rerank-large-v2 using a broad and retrieval-focused dataset mixture. The conclusion reports state-of-the-art results at every released size, with the 400M model within 0.0024 and the 1B model within 0.0001 of the 1.54B teacher on MTEB, while all models outperform the ms-marco-MiniLM-L*-v2 family on MTEB and NanoBEIR.
PaddleOCR 3.5: Running OCR and Document Parsing Tasks with a Transformers Backend
PaddleOCR 3.5 adds a more flexible inference-engine interface for OCR and document parsing, allowing supported models to run with a Hugging Face Transformers backend selected through engine="transformers". It keeps pipelines managed by PaddleOCR while exposing backend-specific settings such as dtype, device placement, and attention implementation through engine_config. The release supports command-line and Python usage, with examples configuring GPU execution and float32, bfloat16, or SDPA options according to hardware and deployment environment. This provides a more natural path for RAG, Document AI, search, analytics, and agent stacks centered on PyTorch and Transformers, while the default paddle_static backend is usually recommended when maximizing OCR or document parsing throughput is the priority.
Granite Embedding Multilingual R2: Open Apache 2.0 Multilingual Embeddings with 32K Context — Best Sub-100M Retrieval Quality
Granite Embedding Multilingual R2 introduces two Apache 2.0 multilingual embedding models built on ModernBERT: a 97M-parameter compact model and a 311M full-size model. Both cover more than 200 languages, receive enhanced retrieval training for 52 languages and nine programming languages, and support context lengths up to 32,768 tokens, compared with 512 for the R1 models. The compact model scores 60.3 on MTEB Multilingual Retrieval, while the full-size model scores 65.2; the reported results place them among the strongest open models in their parameter ranges. R2 replaces the XLM-RoBERTa architecture with alternating attention lengths, rotary position embeddings, newer multilingual and code tokenizers, and Flash Attention 2.0 support, while the 311M model adds Matryoshka embeddings and both models provide ONNX and OpenVINO weights.
Unlocking asynchronicity in continuous batching
Continuous batching improves GPU utilization with tightly packed batches, but its synchronous loop makes the CPU and GPU take turns, leaving the GPU idle while the CPU prepares each next batch. Profiling 8K-token generation with a batch size of 32 and an 8B model recorded 300.6 seconds, with the GPU idle for 24.0% of runtime. The post builds asynchronous batching by overlapping CPU preparation with GPU computation, using non-default CUDA streams, non-blocking transfers, and synchronization events to coordinate dependencies. In the same experiment, GPU activity rose to 99.4% and total generation time fell to 234.5 seconds, a 22% speedup, without new kernels or model changes while maintaining model accuracy.
Building Blocks for Foundation Model Training and Inference on AWS
The post explains how foundation-model scaling now spans pre-training, post-training, and test-time compute, creating shared infrastructure requirements rather than a single pre-training curve. It examines how AWS infrastructure and open-source tools support these workloads across accelerated compute, networking, storage, orchestration, ML frameworks, and observability. Technical detail includes NVIDIA H100, H200, B200, and B300 GPU characteristics, NVLink/NVSwitch scale-up communication, and EFA-based scale-out networking for distributed collectives. The architecture also incorporates EC2 P instances, tiered storage, Slurm, Kubernetes, SageMaker HyperPod, CUDA, NCCL, PyTorch, Prometheus, and Grafana. The conclusion presents integration points between these layers as the basis for diagnosing bottlenecks and making scaling decisions across the foundation-model lifecycle.
vLLM V0 to V1: Correctness Before Corrections in RL
PipelineRL migrated its rollout inference engine from vLLM V0 (0.8.5) to V1 (0.18.1), but trainer-side logprobs, policy ratios, KL, entropy, clip rate, and reward diverged from the V0 reference. The investigation separated semantic mismatch, inference-path mismatch, and objective mismatch, then prioritized backend parity before changing the RL objective. Fixes included processed rollout logprobs, explicit V1 runtime settings disabling prefix caching and async scheduling, a V0-like inflight weight-update path, and an fp32 lm_head for the final projection. With those changes, the final V1 run tracked the V0 trajectory across the reported metrics, while processed_logprobs alone did not resolve the training gap.
Adding Benchmaxxer Repellant to the Open ASR Leaderboard
The Open ASR Leaderboard is adding private English speech-recognition datasets from Appen Inc. and DataoceanAI to evaluate scripted and conversational speech across several accents. To reduce benchmark-specific optimization (“benchmaxxing”) and test-set contamination, the datasets remain private, while the default Average WER continues to use public datasets only. A toggle exposes the private data, with macroaverages for overall, scripted, conversational, American, and non-American subsets; individual split scores are withheld to discourage provider- or accent-specific optimization. The leaderboard uses a shared Hub dataset, normalized transcripts and outputs, and open-source evaluation code, while private evaluation is run after public-set verification when models are submitted.
Granite 4.1 LLMs: How They’re Built
Granite 4.1 is a family of dense decoder-only language models in 3B, 8B, and 30B sizes, trained on approximately 15T tokens. Its five-phase pre-training pipeline moves from broad web-scale data to progressively curated mixtures, increases math and code emphasis, and extends context from 4K to 512K tokens through staged long-context training. Supervised fine-tuning uses about 4.1M curated samples evaluated with LLM-as-Judge and rule-based checks, while reinforcement learning uses on-policy GRPO with DAPO loss. The 8B instruct model matches or surpasses the earlier Granite 4.0-H-Small 32B-A9B MoE model despite its simpler dense architecture and fewer parameters, and the Granite 4.1 models are released under Apache 2.0.
DeepInfra on Hugging Face Inference Providers 🔥
DeepInfra is now a supported Inference Provider on the Hugging Face Hub, adding its serverless inference platform and catalog of more than 100 models to model pages and client integrations. The initial integration supports conversational and text-generation tasks, with access to open-weight LLMs including DeepSeek V4, Kimi-K2.6, and GLM-5.1; support for text-to-image, text-to-video, embeddings, and other tasks will roll out soon. Users can configure provider API keys and preferences in account settings, or authenticate with a Hugging Face token so requests are routed through Hugging Face, including from the Python and JavaScript SDKs and supported agent harnesses. Direct requests are billed by the provider, while routed requests use standard provider API rates without additional markup, and Hugging Face PRO users receive $2 of monthly Inference credits.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.