Activation Steering: One Dial, Two Costs
Activation steering edits a model's internal activations to add or remove a concept at inference time. This explainer maps its one control knob, its two costs, and when it beats prompting.
Loading…
Topic
LLMs
673 posts about LLMs. Every summary links to the original.
- 673 posts
- 49 companies
- 5 explainers
See the idea
LLMs explainers
Antibodies as a Prediction Pipeline: Where AI Models Break Down
Antibody discovery is a three-stage prediction pipeline, and AI models that excel on familiar targets degrade sharply on unfamiliar ones. This explainer maps where that gap appears and what it costs.
Agent Memory: What Outlives the Run
Agent memory is the knowledge that survives a run, not the working state inside one. This explainer separates the two, then maps the write path, the read path, and the database limits that cap recall.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
MongoDB.local NYC 2025: Defining the Ideal Database for the AI Era
At MongoDB.local NYC, MongoDB introduced new database capabilities, AI integrations, and modernization tools designed for agentic and generative AI workloads. The release of MongoDB 8.2 arrives alongside public previews of Search and Vector Search for both MongoDB Community Edition and Enterprise Server. MongoDB also highlighted Voyage AI embedding models and rerankers to improve retrieval accuracy across raw data, metadata, and embeddings. To address the costs and constraints of legacy infrastructure, the new MongoDB Application Modernization Platform combines AI-assisted tooling and specialized expertise to migrate legacy systems. According to MongoDB, early migrations using this platform run two to three times faster, while code rewriting tasks accelerate by an order of magnitude.
MongoDB.local NYC 2025: Definiendo la base de datos ideal para la era de la IA
At MongoDB.local NYC 2025, MongoDB introduced new capabilities and product updates tailored for artificial intelligence workloads. MongoDB 8.2 was unveiled alongside public previews of full-text and vector search for both MongoDB Community Edition and Enterprise Server. The company also detailed its integration with Voyage AI embedding and reranking models to deliver precision retrieval across raw data, metadata, and vectors. To address legacy infrastructure migration hurdles, MongoDB launched the Application Modernization Platform, which combines AI-assisted tooling and specialized workflows to accelerate legacy application refactoring two to three times faster. These architectural updates aim to position the JSON document model as the persistent memory and context store required by autonomous, agentic AI workflows.
MongoDB.local NYC 2025 : définir la base de données idéale à l'ère de l'IA
At the MongoDB.local NYC 2025 event, MongoDB unveiled several platform updates aimed at modernizing enterprise database architectures and supporting emerging agentic artificial intelligence workloads. The newly released MongoDB 8.2 version provides enhanced performance alongside expanded developer features across the broader document database ecosystem. The company also integrated Voyage AI embedding and reranking models to improve context retrieval precision and efficiency when building production-grade AI systems. Furthermore, dedicated Search and Vector Search capabilities have reached general availability across both MongoDB Community Edition and Enterprise Server environments. To streamline infrastructure transitions, the newly introduced Application Modernization Platform leverages AI-powered tooling and specialized techniques to migrate legacy database systems two to three times faster.
MongoDB.local NYC 2025: Definindo o Banco de Dados Ideal para a Era da IA
At MongoDB.local NYC, new product capabilities were unveiled to support modern enterprise workloads and artificial intelligence applications. The release of MongoDB 8.2 delivers improved performance alongside embedding and reranking models from Voyage AI for AI application development. Additionally, search and vector search capabilities have entered public preview for both MongoDB Community Edition and Enterprise Server across customer runtime environments. To address the high operational costs of legacy database systems, MongoDB introduced the Application Modernization Platform to accelerate migration and automated code rewriting using AI-driven tooling. These capabilities position the JSON-based document database as a foundational data layer providing persistent state, contextual retrieval, and memory for agentic AI systems.
MongoDB.local NYC 2025: AI 시대를 위한 이상적인 데이터베이스 정의
MongoDB presented multiple updates at MongoDB.local NYC 2025 aimed at supporting modern AI workloads and enterprise modernization. The event detailed MongoDB 8.2 alongside Voyage AI embedding models and rerankers designed to enhance accuracy and efficiency in AI applications. Search and vector search capabilities have expanded into public preview for MongoDB Community Edition and Enterprise Server. Furthermore, the company launched the MongoDB Application Modernization Platform, which combines AI-driven tools and expert workflows to migrate legacy systems to MongoDB two to three times faster. MongoDB positions its JSON document model as a foundation providing memory, facts, and state continuity for agentic AI applications.
MongoDB.local NYC 2025: Definition der idealen Datenbank für das KI-Zeitalter
MongoDB unveiled several core database updates aimed at supporting artificial intelligence workloads during MongoDB.local NYC 2025. The release of MongoDB 8.2 introduces performance enhancements alongside Voyage AI embedding models and rerankers for AI applications. Furthermore, Search and Vector Search are now available in public preview for both MongoDB Community Edition and MongoDB Enterprise Server. To address the costs and operational friction of legacy systems, MongoDB launched the Application Modernization Platform (AMP), which combines AI-powered tools with specialized talent to accelerate database migrations. These capabilities position the JSON-based document model to supply the persistent state, memory, and retrieval mechanisms required by emerging agentic AI architectures.
MongoDB.local NYC 2025: Definire il database ideale per l'era dell'AI
MongoDB announced several product releases and platform updates aimed at supporting artificial intelligence and agentic workflows at MongoDB.local NYC. The release of MongoDB 8.2 arrives alongside integrations with Voyage AI embedding and reranker models designed to enhance data retrieval precision. In addition, MongoDB launched Search and Vector Search in public preview for both Community Edition and Enterprise Server deployments, bringing vector capabilities to self-managed environments. To assist organizations transitioning away from rigid legacy database systems, MongoDB also introduced the Application Modernization Platform, which combines AI-driven tooling and specialized migration expertise. Early benchmarks from the modernization platform demonstrate legacy migrations running two to three times faster while accelerating code rewriting tasks by an order of magnitude.
Celebrating Excellence: MongoDB Global Partner Awards 2025
MongoDB announced the recipients of its 2025 Global Partner Awards, recognizing cloud providers, systems integrators, and technology vendors for driving enterprise modernization and artificial intelligence adoption. Microsoft earned Global Cloud Partner of the Year for joint integrations linking MongoDB Atlas on Azure with native Microsoft services across healthcare, telecommunications, and financial services. Amazon Web Services received the Global AI Cloud Partner award, highlighted by a joint deployment with Novo Nordisk that reduced a key workflow from twelve weeks to ten minutes using Amazon Bedrock and Atlas. Confluent was named Global Tech Partner of the Year with over 550 joint customer deployments focused on event-driven streaming and multi-agent systems alongside LangChain. Additional honorees included Google Cloud, Accenture, BigID, Pureinsights, gravity9, IBM, and Alibaba Cloud for their contributions across public sector solutions, database-as-a-service offerings, and generative AI frameworks.
Democratizing AI Safety with RiskRubric.ai
Cloud Security Alliance and Noma Security introduced RiskRubric.ai to provide standardized, transparent risk assessments across the open AI model ecosystem. The framework evaluates AI models across six pillars—transparency, reliability, security, privacy, safety, and reputation—using over 1,000 reliability tests, 200 adversarial security probes, automated code scanning, and harmful content evaluations. Each model receives 0–100 scores and A–F letter grades, supplemented by specific vulnerability findings and recommended mitigation strategies to assist deployment filtering. Initial benchmark results across models showed composite scores ranging from 47 to 94 with a median of 81, revealing polarized safety distributions and indicating that security hardening directly correlates with reduced safety risks.
Public AI on Hugging Face Inference Providers 🔥
Hugging Face has integrated Public AI as a supported Inference Provider on the Hugging Face Hub. Public AI operates as a nonprofit, open-source project providing access to sovereign and public models from institutions such as the Swiss AI Initiative and AI Singapore. Its distributed infrastructure combines a vLLM-powered backend serving OpenAI-compatible APIs across partner-donated clusters with a global load-balancing routing layer. Users can access these models through the Hugging Face web UI, Python client SDK, and JavaScript SDK using either direct provider API keys or routed Hugging Face tokens. At the time of announcement, inference through the Public AI provider is free of charge, supported by donated GPU time and advertising subsidies.
Visible Watermarking with Gradio
Generative artificial intelligence tools produce images, video, audio, and text that have become increasingly difficult to distinguish from real-world media captures. To improve synthetic media transparency, Hugging Face introduced visible watermarking capabilities directly into the Gradio web application framework. Developers can now overlay visual watermarks on image and video outputs by specifying a single watermark parameter using file paths, open image objects, or NumPy arrays. The implementation also supports QR code watermarks, which can match the visual style of generated media while conveying detailed background information. Furthermore, the Gradio Chatbot component accepts a text watermark parameter that automatically attaches attribution metadata whenever users copy generated responses to their clipboards.
Introducing the Palmyra-mini family: Powerful, lightweight, and ready to reason!
WRITER released three open models in the Palmyra-mini family ranging between 1.5B and 1.7B parameters, designed for efficient inference and specialized reasoning tasks. Built on the Qwen architecture, the release includes the standard palmyra-mini base model alongside two Chain of Thought variants, palmyra-mini-thinking-a and palmyra-mini-thinking-b. In benchmark evaluations, palmyra-mini achieved 52.6% on Big Bench Hard, while palmyra-mini-thinking-a reached 82.87% on GSM8K and palmyra-mini-thinking-b reached 92.5% on AMC23. For palmyra-mini-thinking-b, applying reinforcement learning fine-tuning to an OpenReasoning-Nemotron-1.5B base improved single-shot pass@1 accuracy while reducing sampling diversity and majority@64 performance. The models are available in GGUF and MLX-BF16 quantizations and support inference engines including vLLM, SGLang, TRTLLM, and TGI.
Tricks from OpenAI gpt-oss YOU 🫵 can use with transformers
To support OpenAI's GPT-OSS series of models, the transformers library introduced several performance upgrades that apply across supported architectures. The release integrates zero-build custom kernels downloadable directly from the Hub, reducing external dependency bloat and compilation friction for operations like Liger RMSNorm and MegaBlocks MoE. Native support for MXFP4 quantization groups vector elements into 32-value blocks with shared scales, allowing GPT-OSS 20B to fit in roughly 16 GB of VRAM and GPT-OSS 120B in roughly 80 GB. In addition, transformers incorporates Flash Attention 3 with attention sinks, continuous batching via the generate_batch API for experimentation, and automatic memory pre-allocation to speed up model loading on GPUs.
Fine-tune Any LLM from the Hugging Face Hub with Together AI
Together AI and Hugging Face have introduced an integration enabling developers to fine-tune compatible Hugging Face Hub models directly on Together AI's managed infrastructure. To launch a fine-tuning job via the Python SDK, users provide a base model from Together's catalog as a configuration template alongside the target Hugging Face repository identifier. This base model template dictates GPU allocation, memory configuration, training pipelines, and inference setup for custom models with matching architectures and sizes. The workflow operates bidirectionally, pulling from public or token-authenticated private repositories and optionally pushing completed checkpoints back to the Hub upon completion. Geared toward CausalLM models under 100 billion parameters, the capability enables faster iteration cycles and domain adaptation without custom DevOps infrastructure.
Jupyter Agents: training LLMs to reason with notebooks
Small language models often struggle to compete with large frontier models on complex, agentic data science tasks. To improve notebook-based reasoning, researchers simplified agent scaffolding down to roughly two hundred lines of code with dedicated execution and final answer tools, boosting baseline easy accuracy on the DABStep benchmark from 44.4 percent to 59.7 percent. They constructed a curated training dataset by deduplicating two terabytes of Kaggle notebooks, automatically fetching five terabytes of linked datasets, and scoring educational value and relevance using Qwen3-32B. Question-answer pairs grounded in verified execution traces were generated to fine-tune compact Qwen3-4B thinking and instruct models. The team released the trained models alongside the dataset and execution sandboxes, establishing a foundation for reinforcement learning and distillation on notebook workflows.
Building Etsy Buyer Profiles with LLMs
Etsy is experimenting with large language models (LLMs) to create anonymous buyer profiles from browsing and purchasing behavior, capturing interests such as styles, categories, and shopping missions. The pipeline combines recent searches, item views, purchases, favorites, session data, and listing information from internal feature-store and BigQuery sources, then prompts an LLM to generate structured profiles with explanations and confidence information. To scale processing, Etsy moved listing data to clustered and partitioned BigQuery tables, shortened input history from two years to nine months, increased batch sizes and compute, added managed concurrency, and used Airflow to stagger tasks. For 10 million users, these changes reduced generation time from 21 days to three and cut estimated cost per million users by 94%; experiments apply profiles to query rewriting and refinement pills.
Building production-ready agentic systems: Lessons from Shopify Sidekick
Shopify’s Sidekick evolved from a simple tool-calling system into an agentic assistant that helps merchants manage stores through natural language, exposing scaling challenges in architecture, evaluation, and training. As its inventory grew beyond 50 tools, the system prompt accumulated conflicting special cases and became difficult to maintain; JIT instructions instead return relevant guidance with tool data. For evaluation, the team uses Ground Truth Sets based on production distributions, human labels from at least three product experts, statistical agreement measures, specialized LLM judges calibrated against humans, and merchant simulation. Judge correlation rose from Cohen’s Kappa 0.02 to 0.61 against a human baseline of 0.69, while syntax validation improved from about 93% to 99% after reward-hacking fixes. GRPO training exposed opt-out, tag, and schema hacking, and the post recommends modularity, procedural plus semantic validation, user simulation, and iterative judge refinement for production agentic systems.
Build Better Agents With MorphLLM
The post addresses a bottleneck in AI agents that write or edit code: correcting a small change can trigger a full-file rewrite, consuming tokens and causing formatting failures, silent omissions, and unreliable results. MorphLLM’s Morph Fast Apply is presented as a semantic, structure-aware edit merge tool: an agent emits small edit_file snippets with //...existing code... placeholders, while the Apply API merges them against the codebase context before the edited file is written back. The post reports over 4,500 tokens per second, 98% accuracy in about six seconds per file, and about four seconds for files up to 9k tokens, compared with 35 seconds and 86% accuracy for traditional search-and-replace with error corrections. It also states that MorphLLM is API-compatible with OpenAI, Vercel AI SDK, MCP, and OpenRouter, and can run in the cloud, self-hosted, or on-premises.
Context engineering case studies: Etsy-specific question answering
The post examines whether prompt engineering can provide truthful Etsy-specific answers for AI-assisted onboarding without fine-tuning an LLM, using Travel & Entertainment (T&E) policy questions and an external Etsy community-forum use case. For the internal pilot, T&E documents were indexed, embedding-based search retrieved relevant sections, and those sections were added to each question; a manually curated 40-question evaluation found approximately 86% correct or satisfactory answers, with 14% wrong or misleading. Explicit uncertainty instructions sometimes avoided hallucinations but also missed available answers, while asking for an explanation or chain-of-thought reasoning could improve results, though complex reasoning required careful setup. The experiments conclude that prompt engineering is a practical alternative to parameter-based fine-tuning, but requesting specific source snippets remains important for flagging potential hallucinations.
The Future Isn't Model Agnostic
The post argues that AI product teams should stop treating model agnosticism as a default priority because users value reliable, familiar behavior more than model flexibility. It says improving models are converging, while differentiation shifts to product experiences shaped by prompts, edge-case handling, workflows, and model-specific behavior. Swapping models therefore means more than changing an endpoint: teams must rewrite prompts, rerun evaluations, and absorb changes users notice in how the product feels. Claude Code and Qwen 3 Coder, which the post says outperforms Claude in benchmarks, illustrate how tightly model behavior, interface, rituals, trust, and expectations can become coupled. The recommended approach is to choose one model, specialize deeply, and treat evaluation as architecture, with games such as an AI Town environment offered as a way to test models.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.