Activation Steering: One Dial, Two Costs
Activation steering edits a model's internal activations to add or remove a concept at inference time. This explainer maps its one control knob, its two costs, and when it beats prompting.
Loading…
Topic
LLMs
648 posts about LLMs. Every summary links to the original.
- 648 posts
- 49 companies
- 5 explainers
See the idea
LLMs explainers
Antibodies as a Prediction Pipeline: Where AI Models Break Down
Antibody discovery is a three-stage prediction pipeline, and AI models that excel on familiar targets degrade sharply on unfamiliar ones. This explainer maps where that gap appears and what it costs.
Agent Memory: What Outlives the Run
Agent memory is the knowledge that survives a run, not the working state inside one. This explainer separates the two, then maps the write path, the read path, and the database limits that cap recall.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Open Responses: What you need to know
Open Responses is a new open inference standard initiated by OpenAI, developed by the open source AI community, and backed by Hugging Face. While agentic workloads require multi-step planning and tool execution, the ecosystem has remained constrained by the legacy Chat Completion format and proprietary Responses implementations. Open Responses addresses this mismatch by standardizing semantic event streaming, provider-specific configuration options, and distinct roles for model providers and intermediary routers. The specification formalizes reasoning visibility across raw traces, protected encrypted content, and sanitized summaries while supporting provider-managed sub-agent execution loops. Developers can now orchestrate multi-step tool calls and stream reasoning deltas across heterogeneous providers through a single consistent API schema.
AI in Investment Management: 2026 Outlook (Part I)
Senior leaders and researchers at Two Sigma analyze the expanding role of artificial intelligence across quantitative investment management for 2026. Rapid model improvements are inverting traditional quantitative research workflows by vastly expanding hypothesis generation and shifting operational bottlenecks toward rapid evaluation. Rather than relying on large language models to execute trades independently, firms are integrating agentic workflows as an underlying operating system across data pipelines and portfolio rooms. However, autonomous agents present alignment challenges because they optimize proxy objective functions relentlessly without intrinsic contextual awareness. Sustained success relies heavily on human supervision, rigorous safety monitoring, and disciplined research execution rather than mere compute scale or model complexity.
Vision RAG: Enabling Search on Any Documents
Traditional retrieval-augmented generation relies on text parsing and optical character recognition to extract data from complex visual documents. These legacy techniques introduce high engineering effort, inconsistent accuracy, and heavy scaling costs. Vision RAG resolves these challenges by embedding entire visual documents directly through multimodal models like Voyage AI's voyage-multimodal-3.5, removing the need for text extraction pipelines. In this pipeline, dense vector embeddings map both text queries and visual assets into a unified representation space. At query time, the system retrieves relevant document images and supplies them alongside the text prompt to Anthropic's Claude 3.5 Sonnet to generate visually grounded responses.
Building the Universal Commerce Protocol
Universal Commerce Protocol (UCP) is an open standard co-developed by Shopify and Google to let AI agents discover merchant capabilities, negotiate what both sides support, and transact with merchants. It addresses commerce complexity arising from cart-, buyer-, and market-dependent payment rules, discount combinations, and fulfillment variations. UCP uses layered extensibility: a shopping service supplies transaction primitives, capabilities such as Checkout and Orders are independently versioned, and extensions add domain-specific schemas without changing the core. Profiles and reverse-domain namespaces support decentralized capability discovery, while checkout states and continue_url enable structured handoff when human input or unsupported capabilities are required. Payment handlers are negotiated dynamically per transaction, and the public protocol is presented as ready for merchants, agents, and providers to build against.
NVIDIA Cosmos Reason 2 Brings Advanced Reasoning To Physical AI
NVIDIA released Cosmos Reason 2, an open reasoning vision-language model designed to improve multi-step planning, uncertainty handling, and physical AI decision-making. Available in 2B and 8B parameter configurations, the model increases input context support to 256K tokens, up from 16K in the prior version. The release adds spatial perception capabilities including optical character recognition, 2D and 3D point localization, bounding boxes, and trajectory generation for robotics. In domain adaptation tests on autonomous vehicle video captioning, fine-tuning the 8B model improved BLEU scores by 10.6% and LingoQA scores by 13.8%. The model leads the Physical AI Bench and Physical Reasoning leaderboards for open visual understanding models.
Introducing Falcon-H1-Arabic: Pushing the Boundaries of Arabic Language AI with Hybrid Architecture
The Technology Innovation Institute introduced Falcon-H1-Arabic, a family of Arabic language models released in 3B, 7B, and 34B parameter sizes. Built on a hybrid architecture, each block runs Mamba State Space Models and Transformer attention in parallel before fusing their representations at the output projection. Pre-training incorporated around 300 billion tokens across Arabic dialects, English, and multilingual content filtered via linguistic analysis. A post-training pipeline combining supervised fine-tuning and direct preference optimization addresses long-context utilization and alignment. Across evaluations on the Open Arabic LLM Leaderboard, the 34B model achieves roughly 75%, exceeding the performance of larger baseline systems like Llama-3.3-70B.
NVIDIA brings agents to life with DGX Spark and Reachy Mini
At CES 2026, NVIDIA presents a desk-based personal assistant that combines DGX Spark processing with a Reachy Mini robot, allowing an agent to converse, perceive camera input, speak, and perform actions. The tutorial assembles NVIDIA Nemotron 3 Nano for reasoning, NVIDIA Nemotron Nano 2 VL for vision, ElevenLabs for text-to-speech, and Reachy Mini hardware or simulation through the NeMo Agent Toolkit and Pipecat. Its configuration routes text, visual, and action requests to different models or a ReAct tool-calling agent, while the robot daemon exposes movement and state capabilities. The guide covers local, cloud, and serverless deployment, then starts the services needed for voice and vision interaction; the completed setup greets users and supports example text and camera prompts.
AprielGuard: A Guardrail for Safety and Adversarial Robustness in Modern LLM Systems
Modern language model systems face security and safety threats spanning prompt injection, jailbreaks, and tool manipulation across long-context agentic workflows. To address these vulnerabilities, researchers introduced AprielGuard, an 8B parameter causal decoder-only transformer based on a downscaled Apriel-1.5 Thinker Base model. The model detects 16 safety risk categories alongside binary adversarial attacks across standalone prompts, multi-turn dialogues, and agentic workflows. Trained on synthetic datasets created with Mixtral-8x7B, NVIDIA NeMo Curator, and SyGra, AprielGuard operates in both explainable reasoning and fast non-reasoning modes. Evaluation shows high precision and recall on safety and adversarial benchmarks, though trade-offs exist between explainability latency and non-reasoning performance.
That’s a Wrap: MongoDB’s 2025 in Review & 2026 Predictions
MongoDB recapped its product releases, executive transitions, and enterprise adoption milestones throughout 2025 alongside predictions for 2026. The company acquired embedding and reranking specialist Voyage AI to mitigate large language model hallucinations and launched MongoDB AMP to accelerate legacy application modernization. Vector search and full-text search capabilities were expanded from MongoDB Atlas to Community Edition and Enterprise Server, enabling on-premises and hybrid retrieval-augmented generation workloads. Enterprise implementations included pharmaceutical distributor McKesson scaling tracking infrastructure to handle 1.2 billion containers annually after replacing its monolithic legacy system. Executive predictions for 2026 emphasized human-in-the-loop verification, grounding language models with factual retrieval, and building multi-cloud resilience against outages and data sovereignty constraints.
Tokenization in Transformers v5: Simpler, Clearer, and More Modular
Transformers v5 overhauls its tokenization framework by separating tokenizer architecture from trained vocabularies. In contrast to v4's dual slow Python and fast Rust files, v5 consolidates each model tokenizer into a single file defaulting to the Rust-backed TokenizersBackend. The pipeline stages—normalizer, pre-tokenizer, model algorithm such as BPE or Unigram, post-processor, and decoder—are now directly exposed and configurable rather than buried in serialized files. Practitioners can instantiate blank tokenizer architectures and train custom vocabularies directly from iterators using native methods like train_new_from_iterator while retaining model-specific formatting rules. The wrapper layer continues to bridge raw tokenization and model requirements by managing chat templates, context limits, and special token insertion.
The Open Evaluation Standard: Benchmarking NVIDIA Nemotron 3 Nano with NeMo Evaluator
Assessing whether large language model improvements stem from genuine advances or underspecified evaluation conditions remains a major challenge across the industry. Most published model evaluations omit critical execution parameters, prompt templates, harness versions, and runtime configurations. In response, NVIDIA released Nemotron 3 Nano 30B A3B alongside its complete, reproducible evaluation recipe built with the open-source NeMo Evaluator library. The library serves as an orchestration layer that unifies diverse benchmark harnesses under standard configurations while decoupling evaluation logic from underlying inference backends. Developers can execute the identical evaluation pipeline against local deployments or hosted endpoints using published YAML configurations and structured logging.
CUGA on Hugging Face: Democratizing Configurable AI Agents
Many existing AI agent frameworks suffer from brittleness, tool misuse, and failures when executing complex workflows. To address these limitations, the open-source Configurable Generalist Agent (CUGA) introduces structured orchestration that decomposes user goals into programmatic subtasks tracked by a dynamic task ledger. The framework integrates agentic patterns like planner-executor and code-act, delegating subtasks to specialized agents that generate pseudo-code before running execution in a secure sandbox. Released under the Apache 2.0 license, CUGA integrates with Langflow for low-code visual workflow assembly and supports multi-tool environments through OpenAPI specs, MCP servers, and LangChain. Testing on inference platforms like Groq with open models such as gpt-oss-120b demonstrates rapid response times during multi-step planning and validation.
New in llama.cpp: Model Management
llama.cpp server now includes a router mode that enables dynamic loading, unloading, and switching between multiple LLMs without restarting the server. The architecture runs each model in an isolated process to ensure a single model crash does not affect other active instances. Running llama-server without specifying a model activates auto-discovery across the cache directory or a designated folder of GGUF files. In addition to on-demand loading and least-recently-used eviction capped by default at four models, the server provides endpoints for manual loading, unloading, and listing model statuses. Models inherit global configuration options or use dedicated configuration presets while also integrating directly into the built-in web UI.
Codex is Open Sourcing AI models
Hugging Face Skills equips AI coding agents like OpenAI Codex to execute end-to-end machine learning workflows. By reading AGENTS.md files and interfacing via the Model Context Protocol, Codex automates dataset validation, training script updates, and job submissions to Hugging Face Jobs. The workflow supports methods including supervised fine-tuning, direct preference optimization, and reinforcement learning for models ranging from 0.5B to 7B parameters. Throughout execution, Codex tracks live metrics via Trackio, records benchmark evaluations against baselines, and maintains Markdown reports. Once training concludes, Codex merges LoRA adapters, applies GGUF quantization, and publishes the resulting models to the Hugging Face Hub for local deployment.
Improving Cursor’s agent for OpenAI Codex models
Cursor describes adapting its agent harness for OpenAI's latest frontier coding model, GPT-5.1-Codex-Max, by pairing model-familiar instructions with Cursor-specific tools and tuning against Cursor Bench. The Codex integration adopts a more shell-forward design, aligns tool names with commands such as rg, explicitly prefers tools over shell commands, and uses sandboxing to limit unauthorized file and network access. It also limits reasoning summaries to one or two sentences, gives literal guidance for invoking read_lints after substantive edits, removes mid-turn user-communication language, and biases the agent toward taking action unless a plan is requested. Preserving Responses API reasoning items is treated as critical: Cursor reports a 30% performance drop for GPT-5-Codex when traces were removed, compared with OpenAI's observed 3% SWE-bench degradation for GPT-5, while prompt-ordering lessons led it to remove wording that discouraged ambitious work.
DeepMath: A lightweight math reasoning Agent with smolagents
Mathematical reasoning in large language models often suffers from lengthy chain-of-thought traces and frequent arithmetic mistakes. To address these issues, DeepMath pairs a Qwen3-4B Thinking base model with a sandboxed Python execution environment built using the smolagents library and vLLM backend. The system offloads deterministic calculations by emitting concise Python snippets, executing them safely with restricted module imports and no network access, and folding returned values back into the context. Training utilizes Group Relative Policy Optimization on the Tool-Integrated Reasoning subset of OpenMathReasoning with temperature scheduling and length constraints. Across benchmarks including MATH500, AIME, HMMT, and HLE, DeepMath reduces output token lengths by up to 66% while increasing overall problem-solving accuracy.
We Got Claude to Fine-Tune an Open Source LLM
Hugging Face Skills equips coding agents like Claude Code, OpenAI Codex, and Google's Gemini CLI to manage end-to-end language model fine-tuning. Using the hf-llm-trainer skill, an agent validates dataset formats, selects cloud hardware, configures authentication, and submits jobs to Hugging Face Jobs. Supported techniques include Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO) for models ranging across various parameter sizes. The integration incorporates Trackio for real-time monitoring and automates LoRA configuration for larger models. Once training completes, the agent pushes artifacts to the Hugging Face Hub and can convert models to GGUF format for local execution.
Streamlining Security Investigations with Agents
Slack's Security Engineering team handles billions of daily security events and needed a reliable way to streamline on-call alert triage. An initial prototype relying on a single 300-word prompt produced inconsistent results and frequently reached spurious conclusions without properly challenging assumptions. To gain precise control, the team decomposed the workflow into chained model invocations with structured JSON outputs organized across three agent personas: a Director, four domain experts, and a Critic. Domain experts gather raw evidence through tool calls, the Critic evaluates finding quality and synthesizes a timeline, and the Director steers investigation phases using tiered model costs. The multi-agent system enables engineers to supervise investigations via a real-time dashboard while uncovering emergent issues like credential exposures across process ancestry chains.
Transformers v5: Simple model definitions powering the AI ecosystem
Transformers v5.0.0rc-0 introduces major architectural updates focused on simplicity, training, inference, and ecosystem interoperability across modern AI workflows. The release adopts a modular modeling approach and centralizes attention implementations into a unified AttentionInterface abstraction to reduce contribution and code review overhead. Support for Flax and TensorFlow is officially sunset in favor of focusing on PyTorch as the primary backend, while tokenization is standardized around the tokenizers library. For execution workloads, v5 adds native continuous batching, paged attention mechanisms, and a dedicated transformers serve OpenAI-compatible serving system. Finally, weight loading is refactored to make low-precision quantization a first-class citizen alongside broad interoperability with formats such as GGUF, MLX, and TorchAO.
Continuous batching from first principles
Large language model serving requires running expensive next-token generation across multiple concurrent user requests. Traditional batching approaches introduce severe padding inefficiencies when mixing variable-length prompts and different generation phases, especially under static shape constraints like CUDA graphs. Continuous batching resolves these inefficiencies by combining key-value caching, chunked prefill, ragged batching, and dynamic request scheduling. Ragged batching eliminates the traditional batch axis by concatenating token sequences into a single tensor and using boolean attention masks to isolate independent sequences. By dynamically removing completed prompts and packing decoding tokens alongside chunked prefill tokens up to a hardware memory budget, serving systems maintain high hardware utilization and throughput.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.