Activation Steering: One Dial, Two Costs
Activation steering edits a model's internal activations to add or remove a concept at inference time. This explainer maps its one control knob, its two costs, and when it beats prompting.
Loading…
Topic
LLMs
670 posts about LLMs. Every summary links to the original.
- 670 posts
- 49 companies
- 5 explainers
See the idea
LLMs explainers
Antibodies as a Prediction Pipeline: Where AI Models Break Down
Antibody discovery is a three-stage prediction pipeline, and AI models that excel on familiar targets degrade sharply on unfamiliar ones. This explainer maps where that gap appears and what it costs.
Agent Memory: What Outlives the Run
Agent memory is the knowledge that survives a run, not the working state inside one. This explainer separates the two, then maps the write path, the read path, and the database limits that cap recall.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
The open ecosystem around ClickHouse
ClickHouse’s open ecosystem spans clients, dashboards, ingestion pipelines, projects built on its engine, and agents that query it, with 2,600+ contributors, 240,000+ commits, and hundreds of millions of downloads cited. It describes open standards including OpenTelemetry for observability, Model Context Protocol (MCP) for AI tooling, and Iceberg and Delta Lake for lakehouse data, plus Iceberg REST catalogs, Apache Polaris, and Unity Catalog. The surrounding stack includes ClickStack, chDB, Langfuse, LibreChat, official language clients, a Kubernetes operator, Postgres bridges such as PeerDB and pg_clickhouse, and integrations with Airflow, dbt, Grafana, Fivetran, and Sigma. Specific updates include full Iceberg v2 support, Iceberg writes in 25.8, Paimon in 25.10, chDB v4’s DataStore API, and a Python client v1.0 with native async and 4x faster cold starts. The post frames this openness as supporting flexible tooling and community-led growth, reporting more than 200 integrations while noting that the ecosystem’s next direction remains unknown.
AI agents are a confused deputy with the keys to your kingdom
Attackers recently compromised over twenty thousand Instagram accounts by manipulating Meta's AI support assistant to rebind recovery email addresses without verifying account ownership. This incident illustrates the classic confused deputy security problem, where a privileged process is persuaded by an unprivileged user to perform unauthorized operations. Because large language model interfaces operate purely on natural language and cannot distinguish instructions from data, the model itself cannot serve as an authorization boundary. Securing AI agents requires verifying caller identity through external policy checks against authenticated sessions rather than relying on chat context or prompt engineering. Teams must enforce least privilege with short-lived scoped credentials, place irreversible actions behind hard policy gates or human approvals, and maintain audit trails of agent actions.
GLM-5.2: Built for Long-Horizon Tasks
GLM-5.2 is presented as a flagship model for long-horizon tasks, extending GLM-5.1’s capability to a solid 1M-token context for sustained engineering work. It was trained extensively on 1M-context coding-agent scenarios involving large-scale implementation, automated research, performance optimization, and complex debugging, and adds effort-level controls to balance capability, latency, and computational cost. Its architecture uses IndexShare, sharing one lightweight indexer across every four transformer layers and reducing per-token FLOPs by 2.9× at 1M context; MTP changes for speculative decoding increase acceptance length by up to 20%. On reported long-horizon and standard coding benchmarks, GLM-5.2 is described as the highest-ranked open-source model: it trails Opus 4.8 by 1% on FrontierSWE, ranks second only to Opus 4.8 on PostTrainBench, and scores 81.0 on Terminal-Bench 2.1 versus GLM-5.1’s 63.5.
Introducing the Agentic CDP: A New Species of CDP for a New Era of Agents
Traditional customer data platforms (CDPs) were built for human-managed, batch-based campaigns, but the post argues that agentic buying requires millisecond speed, hyper-personalization, and richer context. It contrasts the familiar Golden Record with Golden Context, which combines customer data with current business goals and the history and outcomes of prior decisions. The proposed Agentic CDP uses “Infinity Campaigns,” always-on engagement loops that use LLMs and agents to adapt messaging, timing, and channels for individuals. It is also embedded in the data foundation, bringing customer, business, and decision context together under existing governance, and is designed for agents and humans from the outset. Databricks presents CustomerLake as an implementation of these principles for its platform.
Kimi K2.7 Code vs Claude Fable 5: Landing pages that cost 94% less
The experiment compares Kimi K2.7 Code and Claude Fable 5 across 12 landing-page prompts covering B2B SaaS, a rooftop speakeasy, developer tools, and other categories. Kimi cost about 94% less overall, averaging roughly 16 times cheaper than Fable, while producing similar-quality output, especially when given visual references. The team first established a prompt-only baseline, then used a custom design MCP server to provide screenshots, UI elements, and other references directly to multimodal Kimi; this improved hierarchy, typography, composition, readability, loading behavior, and image handling. GPT-5.5 scored screenshots and source code on a 0–100 rubric, and although Fable scored higher, the relatively small quality gap led the authors to consider Kimi’s lower cost a reasonable trade-off for iterative landing-page generation.
Clustering billions of products for agentic commerce with Catalog API
Shopify’s Catalog must reconcile billions of listings whose merchants use different product and variant structures, so AI shopping agents can identify the same real-world product. Product clustering groups related listings under a Universal Product Identifier, with precision prioritized over recall because wrong products can lead to incorrect purchases while missed variants remain findable. The intra-store system starts with rules-based singleton detection, then uses ANN and sparse average linkage for pre-chunking, and applies a two-stage LLM process that proposes clusters and critiques them using the core value proposition framework. Dynamic per-chunk strict JSON schemas enforce complete product coverage, support arbitrary chunking, and improved recall by 8% on the Toloka evaluation dataset in one schema change. The resulting pipeline is presented as a scalable intra-store foundation, while cross-store clustering into global UPIs is identified as the next step.
Building AI Agents for AR Glasses and XR Devices with NVIDIA XR AI
Building AI experiences for augmented reality and wearable devices requires bridging hardware with live media streams, multimodal models, enterprise tools, and runtime infrastructure. NVIDIA XR AI provides an open-source, modular framework connecting extended reality headsets and smart glasses to GPU-accelerated services across cloud, edge, and workstations. In this architecture, camera frames and microphone audio ingest into an XR Media Hub that routes data while keeping raw video pixels in shared memory to minimize overhead. The ecosystem uses NVIDIA Cosmos models for vision-language grounding, NVIDIA Nemotron models for reasoning and tool invocation, and the Model Context Protocol for enterprise integrations. Optional agent orchestration via NVIDIA NeMo Agent Toolkit and spatial streaming through NVIDIA CloudXR support complex workflows across healthcare and manufacturing.
NVIDIA Blackwell Tops MLPerf Training 6.0 with Industry-Leading Scale and Performance
NVIDIA submitted results across all MLPerf Training v6.0 benchmarks, setting performance records on workloads including the DeepSeek-V3 and GPT-OSS-20B Mixture of Experts models. To scale training up to 8,192 Blackwell GPUs, NVIDIA combined hardware cluster designs with scale-out networking via Spectrum-X Ethernet and Quantum InfiniBand. The engineering team eliminated CPU-GPU synchronization bottlenecks in token-dropless MoEs by implementing full-iteration CUDA graphs with synchronization-free operators and paged stashing. Further software optimizations included CuTe DSL kernel fusions, an MXFP8 attention block in the Transformer Engine, and transitioning router elementwise math to FP32. Across these benchmarks, the GB300 NVL72 platform achieved the fastest time to train at scale and the highest normalized per-accelerator performance.
Build On-Device AI Companions with the NVIDIA ACE Game Agent SDK and Unreal Engine 5 Plugins
Game developers integrating on-device artificial intelligence companions face challenges such as conversational latency, state synchronization, and loop prevention. NVIDIA announced the open-source ACE Game Agent SDK alongside a suite of Unreal Engine 5 plugins to streamline native, hardware-accelerated character pipelines. The SDK offers Agent, Chat, and RAG APIs, enabling characters to execute multi-step tool-assisted reasoning and query game databases. Unreal Engine 5 plugins introduce local runtime models including nemo-conformer-ctc-120m for speech recognition, Qwen 3.5 4B for language generation, and Chatterbox Turbo 350M for speech synthesis. Additional tooling expands to motion generation via Animotive Kimodo and rendering updates with the DLSS 4.5 Unreal Engine plugin.
How to Optimize Transformer-Based Models for Low-Precision Training
Accelerating transformer training requires understanding how low-precision formats such as FP8 and NVFP4 impact specific general matrix multiplication (GEMM) workloads. Transformer configurations do not explicitly reveal these shapes, making microbenchmarks necessary before committing to full training runs. NVIDIA Transformer Engine enables quantization and kernel dispatch across precisions, which can be evaluated in realistic autocast mode or kernel-only prequantized mode. Profiling the ESM2-15B model on NVIDIA B300 GPUs demonstrated that NVFP4 achieved a 1.79x blended forward propagation speedup over MXFP8 and up to 4.01x over BF16 in prequantized execution. Although large GEMM dimensions successfully overcome quantization overheads, dynamic scaling, Hadamard transforms, and kernel selection asymmetries moderate real-world gains.
Salesforce signs definitive agreement to acquire Fin
Salesforce has signed a definitive agreement to acquire Fin for approximately $3.6 billion, with closing expected in the fourth quarter of Salesforce’s fiscal year 2027. Fin says it began as Intercom 15 years ago and, nearly four years ago, shifted toward modern LLMs to create and define the Customer Agents category. The companies plan to join forces, with Salesforce’s resources intended to help bring Fin to more consumers, while Fin says little will practically change for customers: the author will remain CEO and Des Traynor will continue running R&D. The announcement also points to recent work including the Apex model and the Operator internal agent, and describes the deal as a pivotal moment for Fin’s employees, founders, and leadership team.
How Wayfair cut ML model costs by 90% (twice!) with Cursor
Wayfair’s Applied Research team used Cursor to automate and parallelize experiments for a product-tag validation model that checked attributes against images, descriptions, and reviews. The model was accurate but too expensive to run across the catalog. In December 2025, five researchers standardized the dataset and evaluation benchmark, then built and tested 110 distinct variants in four days; the winning architecture reduced inference costs by 94%, improved precision, and became the production baseline. In March 2026, the team benchmarked against that baseline, ran 140+ new experiments, and added genetic-algorithm searches, achieving another 90% cost reduction. The workflow used 20+ parallel agents, desktop and CLI surfaces, and cloud agents, leaving researchers to formulate hypotheses, interpret results, and choose candidates while Cursor implemented, ran, and measured variants.
Teaching Sidekick to say no: automated data curation with LLM judge consensus
Sidekick’s customer-segmentation skill was fine-tuned on tens of thousands of successful, de-identified production queries, so it generated zero-result queries instead of refusing impossible requests such as finding customers by occupation. A small Toloka dataset of roughly 600 standard queries and 602 refusal annotations became a seed for four frontier LLM judges, which were calibrated against human-labeled examples and required to agree on both verdict and reasoning. The pipeline used mutually exclusive categories to resolve conflicting labels, filtered disagreements for human review, and fed accepted examples from later production traffic back into fine-tuning. With automated curation, the segmentation pass rate rose from 0.762 to 0.798 versus naive merging, while manual validation measured 86.3% refusal accuracy and a 4.6% false-positive rate. The resulting flywheel is being applied to other Sidekick skill models and data-quality problems.
How Dropbox uses MCP and Dash to close the design-to-code security gap
Dropbox developed a system using the Model Context Protocol (MCP) and Dash's semantic search to bridge the gap between security threat models and code implementation. By retrieving original security documents during pull requests, an LLM agent automatically evaluates whether the proposed code adheres to previously agreed-upon security requirements. This approach surfaces design regressions and missing controls that traditional static analysis tools miss.
Better, faster, less wrong: Enhancing issue grouping
Sentry upgraded its AI-driven issue grouping system to better prevent duplicate issues without merging distinct application errors. The team trained lightonai/modernbert-embed-large using Matryoshka Representation Learning on hundreds of thousands of stacktrace pairs labeled by Claude Sonnet 4.5. To optimize the high-throughput ingestion pipeline, embeddings were truncated from 768 to 64 dimensions, combined with bfloat16 precision, PyTorch SDPA, and CUDA graph compilation. A phased live rollout used threshold-gated index backfilling and automatic fallback to prevent issue spikes during migration. In production, the v2 model cuts the overgrouping rate from 8% to 4%, increases prevented duplicate issues to 70%, and delivers 6x faster inference.
Production-Ready Agents Need A Production-Ready Data Platform
AI development teams face constant shifts in model providers and agent frameworks, demanding data platforms that provide scalable, real-time context management. Agentic workloads require blending unstructured enterprise data, short-term session state, and persistent long-term memory. MongoDB addresses these requirements through its native JSON document model and integrated retrieval capabilities, combining full-text search, vector search, and hybrid search directly over operational data. Customer implementations such as DevRev, ElevenLabs, and Adobe rely on Atlas to achieve sub-100 millisecond hybrid retrieval and handle billions of requests. Additionally, MongoDB is collaborating with LangChain and ecosystem partners to establish open reference architectures and shared interfaces for portable agent memory across frameworks.
How Ecolab rebuilt retail intelligence on Databricks and Anthropic Claude
Ecolab needed to combine audits, health inspections, pest telemetry, and other data from nine systems so retail teams could answer location-specific compliance questions. Its Retail Intelligence application is a native Databricks App using Lakebase Postgres, Lakeflow, and Spark Declarative Pipelines to move governed data into a Unity Catalog lakehouse, while Foundation Model APIs serve Claude Sonnet, Claude Haiku, and Gemini. A Coordinator Agent delegates requests to specialized agents that use Vector Search, SQL, Unity Catalog Functions, and an external MCP server; a Response Agent returns cited answers, with short- and long-term memory stored through Lakebase. The system also applies five Judge LLMs, MLflow tracing, and ai_query() batch inference. Report preparation fell from two weeks to under two minutes, while the assistant supports approximately twelve languages at about 98% accuracy.
Agentic Testing: Where Agents Fit in the E2E Testing Stack
Traditional end-to-end tests validate rigid user journeys, whereas agentic tests verify whether broad goals can be achieved by adapting actions dynamically. To evaluate agentic testing tradeoffs, researchers executed over 200 runs across Playwright Model Context Protocol (MCP), Playwright CLI, and agent-generated Playwright tests using Claude models. Playwright MCP demonstrated high reliability with failure rates of 0% on simple thread replies and approximately 12% on complex search discovery flows. Playwright CLI and generated code struggled more on complex workflows, exhibiting failure rates of approximately 20% and 48% respectively. Although generated tests were faster with average runtimes of roughly three minutes, agentic testing provides a distinct exploratory layer atop deterministic CI test suites.
Making FlashAttention-4 faster for inference
FlashAttention-4, originally designed for pre-training, required modifications to efficiently handle LLM inference workloads dominated by the memory bandwidth-limited decode phase. Modal’s contributions include adjusting parallelism strategies from query parallelism to key/value parallelism, implementing split KV to parallelize across KV tiles, and adding a heuristic for optimal splits. They added support for FP8 inputs, arbitrary KV page sizes via cp.async loads, and reduced wasted work for small query sizes by using a single query tile. The kernel also gained extended GQA packing for irregular head ratios, using cp.async for query tiles to bypass TMA alignment restrictions. These changes, measured in throughput improvements up to 3x for single-token decode and up to 2.4x for small page sizes, demonstrate the leverage of flexible parallelism and tile-based programming for inference optimization.
Catch visual regressions with Snapshots, now in beta
Sentry has released the beta version of Sentry Snapshots to catch visual regressions on pull requests across any platform with a frontend. The tool captures screenshots of an application across various viewports, themes, and languages, compares them against a baseline, and blocks pull requests when diffs are detected. Teams can attach custom context metadata, including test file paths, to help developers and AI agents locate test sources and generate additional test cases. By leveraging the sentry-mcp integration, REST API, and sentry-cli, workflows can retrieve base images, diff masks, and run local diff verification before pushing commits. Originally created at Emerge Tools for mobile applications, this iteration expands automated visual testing capabilities to broader frontend codebases.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.