Activation Steering: One Dial, Two Costs
Activation steering edits a model's internal activations to add or remove a concept at inference time. This explainer maps its one control knob, its two costs, and when it beats prompting.
Loading…
Topic
LLMs
658 posts about LLMs. Every summary links to the original.
- 658 posts
- 49 companies
- 5 explainers
See the idea
LLMs explainers
Antibodies as a Prediction Pipeline: Where AI Models Break Down
Antibody discovery is a three-stage prediction pipeline, and AI models that excel on familiar targets degrade sharply on unfamiliar ones. This explainer maps where that gap appears and what it costs.
Agent Memory: What Outlives the Run
Agent memory is the knowledge that survives a run, not the working state inside one. This explainer separates the two, then maps the write path, the read path, and the database limits that cap recall.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
You can't whisper at an AI agent
Stripe engineers ran roughly a dozen experiments to determine how to get AI agents to use current API versions, follow integration best practices, and load the Stripe skill. Passive hints, including SDK files, AGENTS.md guidance, and API warning responses, were largely ignored, while active prompts and distribution mechanisms performed better. A modular, progressively disclosed skill structure outperformed a monolithic file by roughly 10% and reduced token usage; a CLI login prompt led 30–35% of viewers to copy its install command, and explicit compatibility-mode errors prompted agents to correct version mismatches. The results suggest that agent-facing developer experience is primarily a distribution problem, with hard steering through errors and loaded instructions more reliable than soft hints agents may never encounter.
An Engineer’s Guide to Better AI Skills: Implementing a Testing Process to Optimize Agent…
Software engineers deploying AI agents often encounter unreliable skill invocation for repository-specific architectures, such as Pinterest's iOS framework. To quantify and resolve this issue, a Bash test harness piped positive and negative prompts into Pin-agent and Claude Code while parsing streamed JSON logs for execution signatures. Initial vanilla benchmarks yielded baseline accuracies of 73% for GPT 5.2-codex and 62% for Opus 4.5 Claude. Optimizations such as expanding YAML frontmatter descriptions with architectural context, utilizing capitalized imperative directives, and adding reference tables in AGENTS.md raised invocation rates. Both agents reliably loaded skills when prompts contained explicit invocations, demonstrating that empirical testing paired with descriptive prompting stabilizes agent skill adoption.
Fighting Tool Sprawl: The Case for AI Tool Registries
Rapid enterprise AI agent adoption without centralized infrastructure introduces widespread tool sprawl, duplicated engineering effort, and severe operational opacity. Individual application teams frequently build ad hoc, undocumented integrations, leaving security teams unable to discover, audit, or approve active agent capabilities. Internal enterprise tool registries address this coordination failure by establishing an organization-specific catalog tailored to internal security policies, data, and compliance requirements. A production-grade registry delivers core discovery and version tracking functionality while surfacing security certification metadata and enabling consistent identity-scoped access control across teams. Organizations establishing centralized tool infrastructure can accelerate agent onboarding, implement deny-by-default governance policies, and mitigate technical debt before uncoordinated deployments compound across the enterprise.
Using MemAlign to Improve Evaluation of Traditional Machine Learning in Genie Code
Genie Code’s traditional machine learning workflows are difficult to evaluate because notebook quality depends on code, best practices, and adaptation to data and task. The team built an evaluation pipeline with nine LLM judges, each scoring applicable dimensions—including data exploration, imputation, training, MLflow logging, and cell organization—from 1 to 3, using test cases across classification, regression, and forecasting. On 50 human-graded notebooks, initial judge alignment was weak in several areas, including model training (MAE 0.680) and model use (0.562), with positivity bias contributing to missed technical flaws. Applying MLflow’s MemAlign, which combines generalized semantic guidelines with retrieved episodic examples, reduced judge error by 74–89% on the most important dimensions; removing episodic memory weakened most gains, while data exploration showed a nonsignificant regression.
Pushing the Frontier for Data Agents with Genie
Genie is Databricks’ data agent for complex questions across structured enterprise assets—tables, dashboards, and notebooks—and unstructured sources including workspace files, Google Drive, and Sharepoint. Unlike coding agents operating in static environments, it must discover relevant assets at enterprise scale, determine authoritative knowledge from potentially contradictory sources, and handle questions without verifiable tests or guaranteed answers. It combines specialized knowledge search using semantic context and metadata, parallel thinking across sampled trajectories, and Multi-LLM orchestration with optimized prompts for distinct sub-agents. On an internal benchmark of real-world data-analysis tasks, these techniques raised accuracy from 32% to over 90% against a leading coding agent while reducing cost and latency; specialized search alone improved table-discovery performance by up to 40%, while parallel thinking added latency and token costs before further optimization.
amazon.science
Building trust into AI
Amazon integrates responsible AI practices across four core model development phases: pretraining, post-training, evaluation, and third-party monitoring. During pretraining, researchers augment training corpuses with specialized safety datasets, multimodality alignments, and learning exercises to teach foundational safety concepts instead of purely filtering harmful text. In post-training, reinforcement learning from human feedback uses human preference rankings, auxiliary-reward models, and independent LLM judges to ensure model outputs adhere to safety policies. For specialized use cases requiring access to sensitive domains like security testing, researchers apply low-rank adaptors to alter model behaviors surgically without retraining base weights. Cross-functional policy teams guide these technical stages by mapping risks against eight responsible AI dimensions and continuously updating behavioral boundaries to reflect evolving regulatory frameworks.
Learnings from init to deploy
An agent was used to build a transcription app for a team from initialization through deployment with Stripe Projects, OpenRouter, and Vercel. The app sends video-generated subtitles through an OpenRouter agent with a detailed system prompt and Stripe documentation tool calls, producing a formatted transcription whose speaker and service names are corrected. The author recommends treating agent-generated plans like design documents, verifying provider behavior, and using streaming for long transcripts because Vercel serverless limits made non-streaming responses prone to 504s for a roughly 1,500-line transcript. Testing led to removing an unreliable free-model tier, defaulting to openrouter/auto, and choosing Claude Opus 4.5-thinking over Claude Sonnet 4.5; shared agent files were committed, and a separate review agent was used before human review.
LLM Vs AI: A Practical Guide to Differences, Use Cases, and Tools
This guide distinguishes artificial intelligence, a broad field covering systems that perform tasks associated with human intelligence, from large language models (LLMs), specialized generative AI models for language and code. It places LLMs within generative AI and deep learning, and explains how transformer self-attention processes tokens while training data, parameter scale, and context windows shape capability and limitations. Examples include content drafting, code generation, translation, customer-service chatbots, sentiment analysis, extraction, summarization, and tool-connected agentic workflows, while traditional machine learning remains suited to structured labels and numeric predictions. It recommends retrieval-augmented generation, human review, bias testing, privacy controls, and evidence-based pilots with defined workflows, metrics, budgets, realistic data, and logged outputs.
Generative AI for Business: A Complete Strategy and Implementation Guide
Generative AI is presented as an enterprise technology shift with potential value across customer operations, marketing and sales, software engineering, and research and development, which together are described as accounting for approximately 75% of use-case value across industries. The guide distinguishes generative systems, which create text, images, code, audio, or structured data from predictive models, and explains how foundation models and LLMs support open-ended business applications. Its implementation strategy prioritizes reliable data infrastructure, high-impact low-complexity pilots, and governance covering sensitive data, model limitations, monitoring, human review, and compliance. Recommended execution includes a cross-functional squad, predefined KPIs, user training, baseline measurements, and a 90-day executive review with scale, iterate, or discontinue decisions. Retrieval-augmented generation is identified as the most widely adopted way to reduce hallucinations by grounding responses in verified proprietary data, while ROI is expected within six to twelve months for a well-structured pilot.
AI Applications: Tools, Use Cases, and Platforms
The guide maps AI applications for data scientists, machine learning engineers, and technical leaders, covering predictive AI, generative AI, conversational AI, and autonomous agents across consumer, developer, and enterprise settings. It distinguishes consumer-facing tools from developer platforms and describes production concerns including model lifecycle management, vector search, data lineage, deployment, monitoring, governance, and evaluation. Generative systems create text, images, code, audio, and video from prompts, while large language models and mixture-of-experts architectures are presented as important foundations for enterprise applications; open models offer control over weights, governance, and deployment. The guide recommends defining use cases, assessing data readiness, and building privacy, bias-auditing, and monitoring controls before production, while noting that agents coordinate multi-step workflows across tools, APIs, and databases.
Agentic AI vs Generative AI: Comparing Autonomy, Workflows, and Use Cases
Agentic AI and generative AI differ primarily in whether a system autonomously pursues a multi-step goal or produces content in response to a prompt. The post defines agentic systems through a perceive-plan-act cycle: agents maintain memory and state, decompose goals, call tools or sub-agents, evaluate conditions, and recover from errors, while generative AI typically performs bounded, reactive inference. It presents workflow examples such as sales follow-up and market-intelligence summarization, showing how APIs connect agents to CRMs, databases, communication platforms, and news services while LLMs provide text generation or reasoning at individual steps. RAG can ground generative outputs in external knowledge, but agentic deployments add operational concerns around repeated inference loops, human oversight, provenance logging, and access controls. The conclusion recommends choosing by task structure: generative AI for single-turn creation or summarization, agentic AI for autonomous coordination, and both together for complex enterprise workflows.
Goodbye limitations, hello data: How Qonto is rethinking observability with ClickHouse Cloud
Qonto struggled with slow queries, system instability, and aggressive data sampling when handling distributed traces in Grafana Tempo. To enable wide-event querying without strict cardinality limits, the team migrated their telemetry storage to ClickHouse Cloud. Telemetry from applications, Kubernetes, and GitHub routes through OpenTelemetry collectors over AWS PrivateLink into ClickHouse, while Grafana remains the primary dashboard interface. By compressing 231 TB of uncompressed trace attributes down to 376 GB, Qonto achieved substantial storage savings and integrated an AI incident companion using the ClickHouse Model Context Protocol. The new architecture eliminated cardinality policing, allowed product and support teams to self-serve incident investigations, and enabled real-time streaming pre-aggregations using Apache Flink.
Granite 4.1 LLMs: How They’re Built
Granite 4.1 is a family of dense decoder-only language models in 3B, 8B, and 30B sizes, trained on approximately 15T tokens. Its five-phase pre-training pipeline moves from broad web-scale data to progressively curated mixtures, increases math and code emphasis, and extends context from 4K to 512K tokens through staged long-context training. Supervised fine-tuning uses about 4.1M curated samples evaluated with LLM-as-Judge and rule-based checks, while reinforcement learning uses on-policy GRPO with DAPO loss. The 8B instruct model matches or surpasses the earlier Granite 4.0-H-Small 32B-A9B MoE model despite its simpler dense architecture and fewer parameters, and the Granite 4.1 models are released under Apache 2.0.
Evaluating AI at Scale: How Thumbtack Approaches Reliability, Safety, and Quality in GenAI
Thumbtack evaluates generative AI features across customer and service professional experiences to prevent unsupported claims, tone inconsistencies, and safety issues. The engineering organization transitioned from decentralized evaluation efforts to a dedicated cross-functional Evals team that provides shared infrastructure while preserving product team autonomy. System architectures combine rule-based checks, AI-as-a-judge scorers powered by DeepEval, trust and safety assessments, and crowdsourced human reviews. Teams leverage three parallel orchestration paths, including an MLflow-tracked stack, nightly Databricks batch jobs integrated with data warehouses, and multi-layer human-in-the-loop pipelines. Future milestones target centralized LLM judge registries, full task journey evaluations, automated judge-writing workflows, and cost-aware evaluation orchestration.
DeepInfra on Hugging Face Inference Providers 🔥
DeepInfra is now a supported Inference Provider on the Hugging Face Hub, adding its serverless inference platform and catalog of more than 100 models to model pages and client integrations. The initial integration supports conversational and text-generation tasks, with access to open-weight LLMs including DeepSeek V4, Kimi-K2.6, and GLM-5.1; support for text-to-image, text-to-video, embeddings, and other tasks will roll out soon. Users can configure provider API keys and preferences in account settings, or authenticate with a Hugging Face token so requests are routed through Hugging Face, including from the Python and JavaScript SDKs and supported agent harnesses. Direct requests are billed by the provider, while routed requests use standard provider API rates without additional markup, and Hugging Face PRO users receive $2 of monthly Inference credits.
From init to deploy: building with agents and Stripe Projects
The post describes an internal agentic transcription app built to reduce the manual effort required to correct auto-generated subtitles for Stripe Developers videos. The app uses a Next.js interface with transcript, chat, and corrected-output panels, while Stripe Projects CLI links Vercel and OpenRouter and populates required environment variables. An LLM client supports selectable models, iterative instructions, and a transcription agent whose system prompt emphasizes preserving transcript structure; the agent can also use Stripe documentation search through MCP. In a benchmark on a 771-line meetup script, Gemini 2.5 Pro reached 99% accuracy, while the tested models reduced editing from more than 30 minutes to just over two minutes of generated edits plus a human check. The reported overall reduction was 4x, with testing costs totaling $2.54.
Introducing NVIDIA Nemotron 3 Nano Omni: Long-Context Multimodal Intelligence for Documents, Audio and Video Agents
NVIDIA Nemotron 3 Nano Omni is introduced as an omni-modal understanding model for document analysis, multiple-image reasoning, automatic speech recognition, long audio-video understanding, agentic computer use, and general reasoning. It extends the Nemotron multimodal line from vision-language processing to text, image, video, and audio, targeting long, dense inputs such as 100-plus-page documents and mixed audio-visual evidence. Its unified encoder-projector-decoder combines a Nemotron 3 Nano 30B-A3B hybrid Mamba-Transformer MoE backbone with C-RADIOv4-H and Parakeet-TDT-0.6B-v2 encoders; dynamic visual resolution, Conv3D tubelets, and Efficient Video Sampling support multimodal context compression. The post reports leading or top results across document, video, audio-video, voice, and ASR benchmarks, alongside up to 9x higher throughput and 2.9x single-stream reasoning speed on multimodal use cases, and provides BF16, FP8, and NVFP4 checkpoints.
Connecting Agents to Decisions
Palantir’s Ontology is presented as a decision-centric architecture for human-agent operations, intended to represent how enterprises make and execute decisions rather than only storing or analyzing data. It combines data, logic, action, and security in a shared, dynamic representation that can connect operational data, computational processes, workflow execution, and policy controls. The data layer includes structured, streaming, edge, unstructured, imagery, and decision data, with decision lineage recording the data version, application, and decision context; logic binding connects heterogeneous business, modeling, planning, and optimization assets for human and agent workflows. The architecture exposes approved actions to agents with configurable human review and guardrails, while capturing decisions as learning material for improving workflows and agentic memory. It concludes that a real-time ontology can anchor AI in enterprise conditions while controlling where automation is allowed.
OpenAI GPT-5.5 + Codex, now available and fully governed on Databricks
Databricks announces native support for OpenAI’s GPT-5.5, making it available for coding workflows with Codex, enterprise agents, document pipelines, and data-driven employee workflows. Access is governed through Unity AI Gateway, which provides permissions and rate limits, configurable guardrails for PII, prompt injection, and content safety, MCP tool-call auditing, failover, and request-level observability for model and Codex interactions. The announcement describes GPT-5.5 use with Genie for natural-language analytics, Agent Bricks Custom Agents for multi-step workflows, and Lakeflow Spark Declarative Pipelines for document-focused GenAI ETL involving summarization, extraction, and classification. GPT-5.5 is available on Databricks across AWS, Azure, and GCP, with entry points including AI Playground, governed Codex endpoints, Agent Bricks, and the Foundation Model API.
DeepSeek-V4: a million-token context that agents can actually use
DeepSeek V4 is presented as an open model family aimed at long-running agentic workloads, with Pro and Flash MoE checkpoints offering 1M-token context windows. Its central architecture alternates Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA), compressing KV entries by 4x and 128x respectively, while using FP8/BF16 storage and FP4 indexing. The post-training design preserves reasoning across user turns when tool calls are present, introduces a dedicated XML tool-call schema, and relies on DSec, a Rust sandbox platform for RL rollouts. At 1M tokens, Pro uses 27% of V3.2's single-token FLOPs and 10% of its KV cache, while Flash uses 10% and 7%; V4-Pro-Max reports 80.6% on SWE Verified and 0.59 MRCR 8-needle retrieval at 1M.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.