---
title: "LLMs"
description: "673 posts about LLMs, summarised, each linking to the original."
---

# LLMs
> 673 posts about LLMs, summarised, each linking to the original.

## Articles

### [DeepMath: A lightweight math reasoning Agent with smolagents](https://yomu.fyi/post/deepmath-a-lightweight-math-reasoning-agent-with-smolagents.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Daniel Fleischer, Moshe Berchansky, Moshe Wasserblat
- Published: Dec 4, 2025
- Type: Problem & solution

Mathematical reasoning in large language models often suffers from lengthy chain-of-thought traces and frequent arithmetic mistakes. To address these issues, DeepMath pairs a Qwen3-4B Thinking base model with a sandboxed Python execution environment built using the smolagents library and vLLM backend. The system offloads deterministic calculations by emitting concise Python snippets, executing them safely with restricted module imports and no network access, and folding returned values back into the context. Training utilizes Group Relative Policy Optimization on the Tool-Integrated Reasoning subset of OpenMathReasoning with temperature scheduling and length constraints. Across benchmarks including MATH500, AIME, HMMT, and HLE, DeepMath reduces output token lengths by up to 66% while increasing overall problem-solving accuracy.


### [We Got Claude to Fine-Tune an Open Source LLM](https://yomu.fyi/post/we-got-claude-to-fine-tune-an-open-source-llm.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: ben burtenshaw, shaun smith
- Published: Dec 4, 2025
- Type: Tutorial

Hugging Face Skills equips coding agents like Claude Code, OpenAI Codex, and Google's Gemini CLI to manage end-to-end language model fine-tuning. Using the hf-llm-trainer skill, an agent validates dataset formats, selects cloud hardware, configures authentication, and submits jobs to Hugging Face Jobs. Supported techniques include Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO) for models ranging across various parameter sizes. The integration incorporates Trackio for real-time monitoring and automates LoRA configuration for larger models. Once training completes, the agent pushes artifacts to the Hugging Face Hub and can convert models to GGUF format for local execution.


### [Streamlining Security Investigations with Agents](https://yomu.fyi/post/streamlining-security-investigations-with-agents.md)
- Company: [Slack](https://yomu.fyi/company/slack.md)
- Author: Dominic Marks
- Published: Dec 1, 2025
- Type: Problem & solution

Slack's Security Engineering team handles billions of daily security events and needed a reliable way to streamline on-call alert triage. An initial prototype relying on a single 300-word prompt produced inconsistent results and frequently reached spurious conclusions without properly challenging assumptions. To gain precise control, the team decomposed the workflow into chained model invocations with structured JSON outputs organized across three agent personas: a Director, four domain experts, and a Critic. Domain experts gather raw evidence through tool calls, the Critic evaluates finding quality and synthesizes a timeline, and the Director steers investigation phases using tiered model costs. The multi-agent system enables engineers to supervise investigations via a real-time dashboard while uncovering emergent issues like credential exposures across process ancestry chains.


### [Transformers v5: Simple model definitions powering the AI ecosystem](https://yomu.fyi/post/transformers-v5-simple-model-definitions-powering-the-ai-ecosystem.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Lysandre, Arthur Zucker, Cyril Vallez, Vaibhav Srivastav
- Published: Dec 1, 2025
- Type: Announcement

Transformers v5.0.0rc-0 introduces major architectural updates focused on simplicity, training, inference, and ecosystem interoperability across modern AI workflows. The release adopts a modular modeling approach and centralizes attention implementations into a unified AttentionInterface abstraction to reduce contribution and code review overhead. Support for Flax and TensorFlow is officially sunset in favor of focusing on PyTorch as the primary backend, while tokenization is standardized around the tokenizers library. For execution workloads, v5 adds native continuous batching, paged attention mechanisms, and a dedicated transformers serve OpenAI-compatible serving system. Finally, weight loading is refactored to make low-precision quantization a first-class citizen alongside broad interoperability with formats such as GGUF, MLX, and TorchAO.


### [Continuous batching from first principles](https://yomu.fyi/post/continuous-batching-from-first-principles.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Rémi Ouazan Reboul, Arthur Zucker, Luc Georges
- Published: Nov 25, 2025
- Type: Explainer

Large language model serving requires running expensive next-token generation across multiple concurrent user requests. Traditional batching approaches introduce severe padding inefficiencies when mixing variable-length prompts and different generation phases, especially under static shape constraints like CUDA graphs. Continuous batching resolves these inefficiencies by combining key-value caching, chunked prefill, ragged batching, and dynamic request scheduling. Ragged batching eliminates the traditional batch axis by concatenating token sequences into a single tensor and using boolean attention masks to isolate independent sequences. By dynamically removing completed prompts and packing decoding tokens alongside chunked prefill tokens up to a hardware memory budget, serving systems maintain high hardware utilization and throughput.


### [Building Deep Research: How we Achieved State of the Art](https://yomu.fyi/post/building-deep-research-how-we-achieved-state-of-the-art.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Michael Griff, Dean Sacoransky, Noah Nefsky
- Published: Nov 24, 2025
- Type: Problem & solution

Building production AI research agents presents challenges around context window pollution, escalating token costs, and architectural brittleness across model updates. To resolve these issues, Tavily rebuilt its deep research system around simplified orchestration, compact tooling, and active context curation. The agent mimics human research workflows by distilling tool outputs into concise reflections for ongoing reasoning, withholding raw retrieved web data until the final deliverable stage. Compared to traditional ReAct propagation architectures where token consumption scales quadratically, this reflection-based approach achieves linear token growth. As a result, the system reduced token consumption by 66% compared to Open Deep Research while reaching state-of-the-art performance on DeepResearch Bench.


### [OVHcloud on Hugging Face Inference Providers 🔥](https://yomu.fyi/post/ovhcloud-on-hugging-face-inference-providers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Gilles Closset, Fabien Ric, Elias Tourneux
- Published: Nov 24, 2025
- Type: Announcement

OVHcloud is now integrated as a supported Inference Provider on the Hugging Face Hub, expanding serverless inference options across model pages and client SDKs. The integration enables access to open-weight models, including gpt-oss, Qwen3, DeepSeek R1, and Llama, using European infrastructure with pay-per-token pricing starting at €0.04 per million tokens. Developers can connect via Python and JavaScript SDKs using either direct custom API keys or automatic routing through Hugging Face tokens. The service supports structured outputs, function calling, multimodal workflows, and embedding models while providing sub-200ms first-token response times. Calls routed through Hugging Face pass through standard provider pricing without markups, and PRO tier subscribers receive monthly inference credits.


### [20x Faster TRL Fine-tuning with RapidFire AI](https://yomu.fyi/post/20x-faster-trl-fine-tuning-with-rapidfire-ai.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Kamran Bigdely, Arun Kumar, Quentin Gallouédec
- Published: Nov 21, 2025
- Type: Announcement

Hugging Face TRL now officially integrates with RapidFire AI to accelerate LLM fine-tuning and post-training experimentation through concurrent configuration evaluation. Instead of executing training configurations sequentially, the platform shards datasets into discrete chunks and cycles multiple model configurations across GPUs at chunk boundaries. The runtime handles automatic checkpointing via a shared-memory-based adapter and model spilling mechanism to maintain consistent training state. Experimenters can monitor live metrics on an MLflow-based dashboard and use Interactive Control Ops to stop underperforming runs or clone and warm-start promising configurations mid-flight. Benchmark results on NVIDIA A100 GPUs demonstrate experimentation speedups ranging from 15x to 20x over sequential runs when training TinyLlama-1.1B and Llama-3.2-1B models.


### [Open ASR Leaderboard: Trends and Insights with New Multilingual & Long-Form Tracks](https://yomu.fyi/post/open-asr-leaderboard-trends-and-insights-with-new-multilingual-long-fo.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Eric Bezzam, Steven Zheng, Eustache Le Bihan, Vaibhav Srivastav
- Published: Nov 21, 2025
- Type: Benchmark

The Open ASR Leaderboard expanded its evaluation framework by introducing dedicated multilingual and long-form transcription tracks across eleven datasets. Benchmarking more than sixty open- and closed-source models from eighteen organizations revealed clear operational trade-offs between accuracy and inference speed. Systems combining Conformer encoders with large language model decoders achieved the lowest word error rates in short-form English speech recognition. In contrast, connectionist temporal classification and token-and-duration transducer decoders yielded tenfold to hundredfold improvements in throughput with only slight accuracy penalties. While closed-source commercial systems maintained performance leads in long-form audio transcription, specialized single-language models consistently outperformed generalized systems supporting hundreds of languages.


### [Introducing AnyLanguageModel: One API for Local and Remote LLMs on Apple Platforms](https://yomu.fyi/post/introducing-anylanguagemodel-one-api-for-local-and-remote-llms-on-appl.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Mattt
- Published: Nov 20, 2025
- Type: Announcement

Apple developers frequently face integration friction when trying to support a mix of local and cloud language models across disparate APIs. To resolve this fragmentation, AnyLanguageModel introduces a unified Swift package that acts as a drop-in replacement for Apple's Foundation Models framework. The package standardizes interactions across local backends such as Core ML, MLX, and llama.cpp alongside cloud providers like Anthropic and OpenAI. To avoid dependency bloat from multi-backend support, the library employs Swift 6.1 package traits so projects only import the specific runtime engines they require. Furthermore, the library extends beyond current Foundation Models limitations by adding prompt image support for vision-language models.


### [Apriel-H1: The Surprising Key to Distilling Efficient Reasoning Models](https://yomu.fyi/post/apriel-h1-the-surprising-key-to-distilling-efficient-reasoning-models.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Torsten Scholak, Oleksiy Ostapenko, Raymond Li, Luke Kumar, Joel Lamy-Poirier
- Published: Nov 19, 2025
- Type: Problem & solution

Retrofitting efficiency into existing large language models often fails when naive distillation disrupts complex multi-step reasoning mechanisms. The Apriel-H1 project addresses this challenge by converting a 15B full-attention reasoning model into a Mamba hybrid using targeted distillation on high-quality SFT reasoning traces rather than diffuse pretraining data. Built on the open-source Fast-LLM framework, the process applies reverse KL divergence alongside a staged layer replacement strategy combining Leave-One-Out analysis and dynamic loss evaluations. Across fifty total blocks, replacing thirty attention layers with Mamba mixers yielded the flagship Apriel-H1-15b-Thinker-SFT checkpoint. This configuration achieved a 2.1x throughput increase with minimal benchmark degradation after 76.8B training tokens.


### [Past, Present, and Future](https://yomu.fyi/post/past-present-and-future.md)
- Company: [Cursor](https://yomu.fyi/company/cursor.md)
- Author: Cursor
- Published: Nov 13, 2025
- Type: Announcement

Cursor has announced a $2.3 billion Series D financing at a $29.3 billion post-money valuation, nearly two years after describing a vision for an AI-assisted code editor that could make programming more helpful, answer codebase questions instantly, and reduce the visibility of source code. The company says the early stages of that vision are taking shape while substantial work remains to realize Cursor’s potential. It reports a team of more than 300 engineers, researchers, designers, and operators, over $1 billion in annualized revenue, millions of developers and major engineering organizations as customers, and in-house models generating more code than almost any other LLMs in the world. Existing investors Accel, Thrive, and Andreessen Horowitz participated alongside new partners Coatue, NVIDIA, and Google, while the funding will support further research and Cursor’s next magical moments.


### [You Should Write An Agent](https://yomu.fyi/post/you-should-write-an-agent.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 6, 2025
- Type: Tutorial

Thomas Ptacek argues that people should build an LLM agent before forming strong opinions about the technology, because implementation reveals how simple and surprising the underlying system can be. The example starts with a Python loop around the OpenAI Responses API, storing user and assistant messages in a context list to simulate conversation with a stateless model. Adding a JSON-defined ping function and a tool-calling loop lets the model choose and execute multiple network checks, demonstrating behavior not explicitly hard-coded in the loop. The discussion extends this pattern to coding agents, contexts persisted in SQLite, sub-agents, and design choices involving nondeterminism, ground truth, interchange formats, and token costs. It concludes that these remain open software-engineering problems whose practical behavior can be understood by building and iterating on agents.


### [How we built a custom vision LLM to improve document processing at Grab](https://yomu.fyi/post/how-we-built-a-custom-vision-llm-to-improve-document-processing-at-gra.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jia Chen
- Published: Nov 4, 2025
- Type: Problem & solution

Grab developed a specialized Vision LLM for extracting text and key information from Southeast Asian identity cards, driver’s licenses, and registration certificates, where varied languages and layouts challenged traditional OCR and existing models. It selected Qwen2-VL 2B for its efficient size, SEA language tokenizer coverage, and dynamic native-resolution image handling, while synthetic Common Crawl text images and Documint supplied reviewed training labels for detection, orientation, OCR, and KIE. LoRA fine-tuning improved Latin-script documents but struggled with Thai, Vietnamese, and dense unstructured layouts, leading to two-stage full-parameter training with synthetic SEA text images followed by task-specific document fine-tuning. The fully fine-tuned 2B model raised Thai accuracy by 70 percentage points and Vietnamese accuracy by 40 points over baseline, while a custom roughly 1B-parameter model combining Qwen2-VL 2B’s vision encoder with Qwen2.5 0.5B’s decoder stayed within a 3-point accuracy gap across most document types and achieved lower latency.


### [Aligning to What? Rethinking Agent Generalization in MiniMax M2](https://yomu.fyi/post/aligning-to-what-rethinking-agent-generalization-in-minimax-m2.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: MiniMax
- Published: Oct 30, 2025
- Type: Problem & solution

MiniMax M2 addresses the performance gap between agent benchmark scores and real-world deployment across diverse scaffolding frameworks and environments. While initial alignment efforts relied on scaling tool variety, models continued to fail when subjected to slight variations in prompts, scaffolding, and tool responses. To resolve this instability, the team introduced interleaved thinking, which enables models to execute internal reasoning at arbitrary steps throughout long-horizon tasks to adapt dynamically to external tool outputs. Additionally, developers constructed a data pipeline designed for full-trajectory generalization by introducing perturbations across system prompts, user objectives, environments, and tool feedback. In internal evaluations, MiniMax M2 maintained robust tool-calling and instruction-following performance when tested against unfamiliar, cold-start agent scaffolding frameworks.


### [On the Shifting Global Compute Landscape](https://yomu.fyi/post/on-the-shifting-global-compute-landscape.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tiezhen WANG, Irene Solaiman
- Published: Oct 29, 2025
- Type: Explainer

United States export controls on advanced artificial intelligence hardware have catalyzed rapid expansion across China's domestic silicon and open-weight model ecosystem. Faced with restricted access to high-end NVIDIA GPUs, Chinese developers accelerated deployments on domestic accelerators, including Huawei Ascend, Cambricon, and Baidu Kunlun. Hardware scarcity spurred architectural and algorithmic innovations in compute efficiency, notably DeepSeek's Multi-head Latent Attention and Group Relative Policy Optimization, alongside substantial post-training cost reductions. Organizations such as Baidu and Ant Group now train foundation models directly on domestic hardware, fostering non-CUDA software stacks and lowering inference costs globally. Consequently, the global artificial intelligence infrastructure is shifting from an exclusively American-focused paradigm toward a dual-ecosystem landscape powered by domestic chips and open-weight architectures.


### [Granite 4.0 Nano: Just how small can you go?](https://yomu.fyi/post/granite-4-0-nano-just-how-small-can-you-go.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Kate Soule, Rameswar Panda
- Published: Oct 28, 2025
- Type: Announcement

IBM has released Granite 4.0 Nano, a collection of sub-billion and ~1.5B parameter language models tailored for edge and on-device deployments. Trained on more than 15 trillion tokens under an Apache 2.0 license, the suite includes four instruct models alongside their base counterparts. The lineup introduces hybrid-SSM architectures—specifically the ~1.5B parameter Granite 4.0 H 1B and the ~350M parameter Granite 4.0 H 350M—as well as standard transformer variants suited for runtimes like llama.cpp. According to benchmark results across general knowledge, math, code, and safety domains, these compact models demonstrated strong capability improvements relative to their parameter footprint. Furthermore, evaluation on IFEval and Berkeley's Function Calling Leaderboard v3 indicated competitive performance on instruction following and tool calling tasks.


### [Platform Thinking: Three Views from Two Sigma Leaders](https://yomu.fyi/post/platform-thinking-three-views-from-two-sigma-leaders.md)
- Company: [Two Sigma](https://yomu.fyi/company/two-sigma.md)
- Author: Emily Majewski
- Published: Oct 23, 2025
- Type: Explainer

Two Sigma structures its quantitative investment operations around foundational platforms that balance operational speed with scientific rigor. In data engineering, foundational teams provide both raw and curated datasets using BigQuery, CI/CD-managed transformation pipelines, and reusable data contracts termed ice cubes to serve ninety percent of use cases. Quantitative modeling incorporates open-source large language models trained strictly on point-in-time data to rapidly generate features while preventing temporal leakage. Long-term platform innovation follows an epsilon and omega strategy that couples overarching vision with iterative, learning-focused steps. Platform leadership categorizes artificial intelligence applications into advisory, oracle, operational, and agentic functions designed to enhance human productivity rather than replace researchers.


### [Building the Open Agent Ecosystem Together: Introducing OpenEnv](https://yomu.fyi/post/building-the-open-agent-ecosystem-together-introducing-openenv.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Joseph Spisak, Davide Testuggine, Zach Wentz, Pierre Andrews, Sanyam Bhutani, Hamid Shojanazeri, Pankit Thapar, Emre Guven, Lewis Tunstall, Vaibhav Srivastav
- Published: Oct 23, 2025
- Type: Announcement

Meta and Hugging Face have partnered to launch OpenEnv Hub, an open platform and specification designed for agentic AI environments. Large language models require external tools and APIs to complete tasks, but exposing broad toolsets directly introduces significant safety and semantic challenges. In response, OpenEnv establishes sandboxed environments providing defined APIs, authenticated tool access, and isolation for reinforcement learning post-training and deployment pipelines. The initial release includes the OpenEnv 0.1 RFC series covering component architecture, packaging, communication, and Model Context Protocol tool encapsulation. Creators can run local Docker-based environments using standard step, reset, and close interfaces while integrating with frameworks like TorchForge, TRL, verl, and SkyRL.


### [Sentence Transformers is joining Hugging Face!](https://yomu.fyi/post/sentence-transformers-is-joining-hugging-face.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tom Aarsen
- Published: Oct 22, 2025
- Type: Announcement

Sentence Transformers is officially transitioning its stewardship from the Ubiquitous Knowledge Processing Lab at TU Darmstadt to Hugging Face. The open-source embedding library was introduced in 2019 by Nils Reimers using a Siamese network architecture to overcome standard BERT limitations on sentence-level semantics. Tom Aarsen of Hugging Face has maintained the project since late 2023 and will continue leading its development. Under Hugging Face, the project retains its Apache 2.0 license and leverages infrastructure such as continuous integration and automated testing. Sentence Transformers currently serves over one million monthly unique users, with more than 16,000 community models publicly hosted on the Hugging Face Hub.


[Newer posts](https://yomu.fyi/topic/llm/page/27.md) · [Older posts](https://yomu.fyi/topic/llm/page/29.md)