Loading…
Hugging Face
AI platform and community providing machine learning models, datasets, libraries, and tools for building AI applications.
- 320 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
How UK AISI and EvalEval Are Making Benchmark Results Reproducible
The UK AI Security Institute (AISI) has begun using EvalEval’s Every Eval Ever (EEE) schema and Evaluation Cards platform to openly share LLM evaluation results. This collaboration, stemming from a joint workshop at NeurIPS 2025, aims to make benchmark results more reproducible by providing standardized configuration and context alongside verified scores. AISI released results for five benchmarks—HealthBench, FrontierMath, Humanity’s Last Exam, SWE-Bench Pro, and Terminal-Bench 2.0—across six frontier models, plus two cyber evaluations using a partially overlapping model set. The release accompanies a paper studying how inference-time compute and evaluation protocol affect performance, revealing that models continued solving tasks when given correctness feedback. By adopting a shared reporting schema, evaluators can compare findings across studies and understand how setup choices influence reported performance, while EvalEval invites model developers, evaluation creators, and researchers to contribute to the shared infrastructure.
Transformers now runs llama.cpp quants
Transformers now supports running GGUF quantized models locally, reusing llama.cpp’s ggml kernels for efficient inference on Apple Silicon. The GGUF format packages weights, tokenizer, and chat template in one file, with quantization levels like Q4_K_M that trade precision for memory. Users can load a checkpoint from the Hub with from_pretrained and generate text using the standard Transformers API, or serve it via an OpenAI-compatible endpoint. The initial focus is on the Qwen3.5 architecture, with benchmarks on a MacBook Pro M2 Max showing token-generation rates close to llama.cpp, including prefill time. The integration uses packed quantization kernels and generation-loop optimizations to reduce overhead, though it is currently limited to single conversations on MPS without batching support.
Jun Kim, oMLX creator and maintainer, joins Hugging Face to support the MLX community
Jun Kim, creator and maintainer of oMLX, has joined Hugging Face to support the MLX community. MLX is Apple’s framework for local AI, optimized for Apple Silicon, and Hugging Face is the Hub for MLX models. oMLX, staying Apache 2.0, will gain stability and faster development as a funded, maintained project, allowing Jun to guide contributors and build for the long term. The goal is to unblock the community to run local AI in any form, with oMLX serving as a testbed leveraging dependencies like mlx-lm and mlx-vlm, and collaborating with projects such as LMStudio. A key focus is streamlining the transition from transformers model definitions to reference MLX implementations for different engines, so each can focus on unique features.
Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem
Removing entire transformer blocks is a cheap way to speed up large language models, yielding predictable inference gains and stacking with quantization and other compression methods. The challenge is that block choices interact, making it a combinatorial problem rather than a simple ranking task. The authors reformulate block selection as a constrained binary optimization (CBO) problem, mapping it onto an Ising glass, a disordered spin system with all-to-all interactions and a fixed number of removed blocks. They note that mean-field methods, which treat each block independently, are inadequate for this interacting system. Block removal composes with quantization, low-rank compression, width pruning, and knowledge-distillation-based healing, fitting into a larger pipeline rather than competing with it. The paper is titled ‘LLM Compression by Block Removal with Constrained Binary Optimization.’
tokenizers v1: encode, decode and scaling, measured
The tokenizers library is being reworked for version 1 with a heavy focus on performance, as tokenization can become a bottleneck that starves models of data when workloads scale. The work draws on ideas from other open-source tokenizers like gigatoken, tiktoken, and kitoken. Key optimizations include using SIMD instructions for UTF-8 text, ensuring merges never cross pre-token boundaries, and reusing scratch buffers to avoid allocations. The release candidate is implemented in the Rust pre-release on crates.io, with the crate split into tk-encode, tk-serialize, tk-convert, and tk-train. Batched model calls and faster decoding with parallel batch processing are also part of the improvements.
Your Agent Aced the Task. Will It Do It Again?
Agent reliability can be obscured by average benchmark scores: Mean@k reports average success, while Pass^k measures tasks succeeding on every repeated run. On AppWorld, a ReAct agent using GPT-4.1 achieved Mean@5 of 77.4% but Pass^5 of 53.0%, leaving a 24.4-point consistency gap, with temperature set to 0.0. The Consistency Analyzer diagnoses flip-prone decision points by resampling each step in one recorded trajectory with k=5 completions, requiring no ground truth, logits, or end-to-end replay. ALTK-Evolve turns flagged instability into reusable consistency guidelines that are retrieved at inference time. In the reported evaluation, these guidelines raised aggregate Pass^5 to 69.0% from 53.0% and Mean@5 to 81.0% from 77.4%, cutting the gap to 12.0 percentage points; similar-task Pass^5 also improved by 13.0pp.
Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL
An AsyncGRPO setup uses LoRA adapter-only synchronization to separate vLLM Jobs, rather than moving full model weights over NCCL. A Storage Bucket mounted at the same path lets the trainer publish versioned adapters that replicas load, while a proxy adds authentication, routes rollouts toward KV-cached prefixes, and broadcasts adapter updates. Versioned adapter names prevent cached prefixes from mixing policy versions, and max_staleness=4 requires six LoRA slots during replacement. Five runs identify client-side concurrency, checkpointing, and packing as important throughput factors: raising in-flight requests to 384, disabling checkpointing, and packing improved median step time from 22.9 to 4.8 seconds. At 500 steps, the final run took 53 minutes instead of 3 hours 27 minutes, trained 31% more samples, and had basically the same reward curve.
Rebuilding AUTOMATIC1111 with Gradio Workflow
Workflow1111 rebuilds most of AUTOMATIC1111’s stable-diffusion-webui feature set as a single Gradio Workflow canvas. It combines eleven media pipelines and 73 nodes for text-to-image, refinement, image-to-image, prompting, interrogation, detection masks, annotators, upscaling, background removal, PNG metadata, and image-to-video. The implementation uses four operator kinds—Python fn functions, model calls through InferenceClient, Space calls, and Hub dataset rows—with parallel branches for independent work; 32 of 36 operator nodes are fn nodes, and 22 run in-process without network calls. Models can run through Inference Providers, Spaces, or a local GPU, while output nodes automatically become REST endpoints and MCP tools. The result is a browser-accessible, OAuth-enabled workflow that can also be duplicated, rewired, deployed, and called from code.
IBM releases SOTA Granite Time Series PatchTST-FM-r2 model with commercial-friendly license
IBM has released Granite Time Series PatchTST-FM-r2, a roughly 385-million-parameter foundation model for zero-shot forecasting across demand, prices, energy loads, traffic, telemetry, and other time series. The model uses conformer blocks combining multi-head self-attention with temporal convolution, 50% overlapping patches with Hamming-window weighting, overlap-and-add forecasting, contexts up to 8,192 steps, and a 99-quantile prediction head for probabilistic outputs and imputation support. On GIFT-Eval, it ranks second for both CRPS and MASE among replicable zero-shot models as of September 8, 2026, and first in that category among permissively licensed models; including pretrained models, it ranks third for CRPS and fourth for MASE. IBM provides open weights, architecture, inference code, and reproducibility code under Apache 2.0 and OpenMDW 1.0. The release also describes Python use and streaming integration through Apache Flink on Confluent Cloud.
Safety for Whom? Refusing the Right Subset of a Topic, Not the Whole Topic
Topic-level safety systems can conflate a dangerous subject with dangerous intent, causing models to refuse legitimate requests alongside harmful ones. The paper studies deployment-specific boundaries using political persuasion as a testbed: refuse manipulation or targeted persuasion while answering factual political questions. Its training pipeline repairs self-generation coverage through escalating retries, adds verified surface-dangerous benign examples, and evaluates held-out harmful-benign prompt pairs rather than relying on aggregate refusal rates. On Qwen3-8B, escalated coverage raises in-distribution political refusal from 9.47% to 84.75%, while the strongest configuration reduces unsafe responses across three broader benchmarks from 26.26% to 0.14%; however, XSTest over-refusal rises to 74.00%. Adding benign boundary data reduces over-refusal on comply-worthy held-out prompts from 32.94% to 4.16%, with harmful-side refusal declining from 91.88% to 87.72%.
NeoMME: an efficient Multimodal-native and Multilingual Encoder
NeoMME is a family of 260M and 800M multilingual multimodal encoders designed to represent text and images without a separate pretrained vision tower or causal language model. A single bidirectional Transformer processes text tokens and raw 32×32 image patches, while pretraining from scratch uses a masked discrete-diffusion text-denoising objective with high masking to encourage image-grounded reconstruction. The retrieval variant adds jointly trained dense mean-pooled and 128-dimensional late-interaction heads, returning both in one forward pass for page-image retrieval. On ViDoRe v3, the 260M model reaches 0.523 and both sizes lie on the model-size Pareto frontier; at 2048×2048, it encodes about 51 pages per second on an NVIDIA L40S, roughly twice ColModernVBERT's throughput. Hierarchical token pooling and asymmetric quantization reduce late-interaction storage from roughly 1.5 MB to 6 kB per page, a 255× reduction, while retaining more than 95% of baseline nDCG@10.
Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps
This guide presents a low-cost recipe for improving structured-output compliance in Liquid AI’s LFM2.5-350M, a task where parseable, schema-adherent responses determine whether an LLM can feed downstream systems. It fine-tunes the model with Group Relative Policy Optimization (GRPO) through TRL, using about 500 samples, 100 steps, three weighted rewards for format, field count, and JSON Schema validation, and a LoRA adapter training roughly 6M parameters. Prompt augmentation covers fenced-code-block instructions and top-level-array tasks, while evaluation uses IFStruct with llama.cpp on an OpenAI-compatible local server. On 2,000 IFStruct samples, the BF16 base model passes 22.6%, rising to 29.7% after tuning. JSON improves from 18.0% to 31.9%, bare-list compliance from 16.6% to 29.7%, while YAML changes little.
Give Your Coding Agents a Memory You Own
funes is presented as a durable memory layer for coding agents that turns existing session traces into searchable, provenance-backed working memory across machines and agent types. A single binary adds recall and get tools to Claude Code, Codex, pi, or Hermes, builds an incremental local index, and automatically indexes completed turns. Its deterministic pipeline normalizes traces, chunks them, embeds them with a pinned local model, stores them in Lance, and combines vector and BM25 search with rank fusion, cross-encoder reranking, recency weighting, and neighboring chunks. Memories can remain local or be bound to a private Hugging Face dataset owned by the user, with credential redaction, secret scanning, local caching, and Hub-provided access control and versioning. In the handoff-vs-recall benchmark, recall was the cheapest option on both tasks, costing 8x less than a written handoff on one and 4x less on the other, while compaction succeeded on only one task.
Training a coding model to paint watercolours with TRL and OpenEnv
The project reproduces an earlier idea: training a language model to write JavaScript that paints watercolours with p5.brush, using TRL and OpenEnv. Its open pipeline includes a hand-rated reference pool, an RL environment, training scripts, trained models, and Hugging Face Jobs, Spaces, Inference Providers, and Hub artifacts. The environment renders sketches in headless Chromium, restricts the model to ten library methods, rejects cheats through a gate, and combines gate, length, pairwise-judge, and HPSv3 rewards. Three reward mixes were trained: judge-led (0.60/0.30), hps-led (0.30/0.60), and hps-only (0.00/0.90); the first two ran to step 110, while validation stopped at step 60. The post reports that both pairwise-judge runs learned and identifies the hand-rated pool as the bottleneck.
Real-Time Intelligence with IBM Time Series Models on Confluent
IBM and Confluent are making IBM Granite Time Series foundation models available in Early Access on Confluent Cloud, bringing forecasting and other time-series capabilities into streaming workflows. The announcement addresses the cost and delay of bespoke models by offering models that generalize across previously unseen series and can be used for forecasting, anomaly detection, similarity search, classification, gap-filling and optimization. Native inference runs directly in Apache Flink through Confluent’s AI_FORECAST and AI_DETECT_ANOMALIES functions, with Flink maintaining keyed, fault-tolerant state for each series and results written to Kafka topics. Confluent manages serving, scaling and runtime operations while retaining schemas, lineage, access controls and replayable data within the platform. Early Access starts on Confluent Cloud on AWS at no charge, with Confluent Platform planned next for on-premises and hybrid environments.
BenchMIRT: What are LLM benchmarks actually measuring?
BenchMIRT is a method for auditing LLM benchmarks at the individual-prompt level, where questions may measure capabilities beyond a benchmark’s stated goal. It applies multidimensional Item Response Theory (MIRT) to model and question results, estimating model abilities plus each item’s difficulty and discrimination across dimensions. Trained on 100 LLMs, 16 benchmarks, and more than 34K questions, it independently recovered safety and general reasoning as dominant dimensions, repeatedly producing the same result. The analysis found, for example, that BBQ and WMDP aligned more strongly with reasoning than safety, while HarmBench combined safety- and reasoning-associated question groups; retaining 10% of items generally preserved capability rankings, and held-out correctness prediction reached 79% versus 70% for a baseline. BenchMIRT’s dimensions depend on the benchmark set, and its item-level transparency can also enable evaluation trimming.
Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI
Hugging Face’s WebAI team is releasing @huggingface/kernels, a JavaScript loader for optimized WebGPU kernels hosted on the Hub, alongside an initial collection of 207 Apache-2.0 kernels. Each kernel repository packages a versioned contract, metadata, correctness cases, benchmark cases, and parameterized WGSL shader templates, while the loader derives outputs and selects variants for a request and device. Fleet adds browser-based correctness and performance testing, with consent-based private evidence used to identify failures, compare variants, and guide optimization. In comparisons against ORT WebGPU on an Apple M4, matching cases showed a 2.57x geometric-mean speedup and a 1.90x median speedup, though the measurements cover individual GPU operations and exclude setup costs. The project is also intended as a shared foundation for browser inference and future integration with higher-level tooling.
The Open ASR Leaderboard Adds Its First Global South Language
Voice Arena and Hugging Face have added two Monsoon evaluation sets, en-IN and hi-IN, to the Open ASR Leaderboard, introducing Indian English and Hindi coverage. The initiative addresses a limitation of aggregate WER: existing test sets record what was said but little about who spoke, making disparities across geography, age, gender, accent, devices, and other conditions difficult to see. The four speaker-disjoint public and private splits contain 4,888 speakers and vary across nine collection axes, using unscripted dual-channel conversations, broad district and device coverage, and 12 speaker metadata attributes. Because Hindi has extensive valid orthographic variation, its references use manually curated lattices and are scored with Orthographically-Informed Word Error Rate (OIWER); comparisons with flattened references show higher, non-uniform error rates and changed system rankings.
Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers
Sentence Transformers v6.0 adds MultiVectorEncoder, a fourth model type for ColBERT-style late-interaction retrieval, alongside a training approach for finetuning existing models or building them from base transformers. Multi-vector models retain one small vector per token and score query-document pairs with MaxSim, where each query token selects its best document-token match and the scores are summed. The guide covers models, datasets, loss functions, training arguments, evaluators, trainers, callbacks, and multi-dataset training, with examples using pip install -U "sentence-transformers[train]". On a medical retrieval evaluation, the finetuned multi-vector-encoder/mLateOn-medical model, trained for 14.5 hours on a single RTX 3090, outperformed every general-purpose retrieval model tested across dense, sparse, lexical, and multi-vector systems.
Granite 4.2 LLMs: How They're Built
Granite 4.2 is IBM’s first family of dense, decoder-only reasoning language models, offered in 3B, 8B, and 30B sizes. The models use a shared design built around Grouped Query Attention, Rotary Position Embedding, SwiGLU, RMSNorm, bfloat16 precision, and separate input/output embeddings; each has a 131,072-token sequence length. Pre-training starts from scratch on approximately 15 trillion tokens across five phases, with the final phase extending the context window to 512K tokens. Supervised fine-tuning uses about 7.2 million samples spanning agentic and non-agentic data, followed by multi-stage reinforcement learning; the 8B and 30B models also receive agentic RL in sandboxed environments for tool use, code execution, terminal control, and web search. All models support thinking, non-thinking, and low-effort thinking modes, native tool calling, and Apache 2.0 licensing.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.