Activation Steering: One Dial, Two Costs
Activation steering edits a model's internal activations to add or remove a concept at inference time. This explainer maps its one control knob, its two costs, and when it beats prompting.
Loading…
Topic
LLMs
648 posts about LLMs. Every summary links to the original.
- 648 posts
- 49 companies
- 5 explainers
See the idea
LLMs explainers
Antibodies as a Prediction Pipeline: Where AI Models Break Down
Antibody discovery is a three-stage prediction pipeline, and AI models that excel on familiar targets degrade sharply on unfamiliar ones. This explainer maps where that gap appears and what it costs.
Agent Memory: What Outlives the Run
Agent memory is the knowledge that survives a run, not the working state inside one. This explainer separates the two, then maps the write path, the read path, and the database limits that cap recall.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
AI and the Future of Cybersecurity: Why Openness Matters
Mythos is presented as a frontier AI system that rapidly finds and patches software vulnerabilities, with the post arguing that its surrounding system matters more than the language model alone. Its recipe combines substantial compute, software-relevant training data, vulnerability-probing and patching scaffolding, speed, and some autonomy; together, these elements can uncover vulnerabilities, find exploits, and build patches. The post argues that open code and tooling can distribute detection, verification, coordination, and patch propagation across communities, while reducing capability asymmetry between attackers and defenders. For defensive use, it recommends semi-autonomous agents with prespecified actions, human approval, auditable logs, and private deployment using open components and existing security tools. This approach is especially relevant to high-stakes organizations that need inspectable, controllable systems and protection for sensitive data.
Ecom-RLVE: Adaptive Verifiable Environments for E-Commerce Conversational Agents
Ecom-RLVE extends the RLVE framework from single-turn reasoning puzzles to multi-turn, tool-augmented e-commerce conversations. EcomRLVE-GYM supplies eight environments spanning product discovery, substitution, cart building, returns, order tracking, policy QA, bundle planning, and multi-intent journeys, with procedural problem generation, a 12-axis difficulty curriculum, and algorithmically verifiable rewards. Each episode requires tool use and world-state changes; rewards combine task completion, efficiency, and a penalty for recommending product IDs not retrieved, while malformed outputs and illegal calls fail immediately. Adaptive scheduling advances each environment independently when its success rate is reliable. An initial study trained Qwen 3 8B with DAPO for 300 steps on cart building and reported progressively increasing difficulty reached, consistent with a steady learning signal.
The PR you would have opened yourself
mlx-lm contributors can use a Skill and a separate test harness to port language models from transformers, with the goal of making new architectures available shortly after their upstream implementations land. The work addresses rising agent-generated pull-request volume and the difficulty of preserving implicit codebase conventions, user contracts, and performance-sensitive behavior that agents may not understand. Given a conversion prompt, the Skill sets up environments, finds model variants and checkpoints, reads transformers modeling code, writes an MLX implementation, and iterates through tests including dtype checks, generation examples, numerical comparisons, and per-layer comparisons. The resulting pull requests disclose agent assistance and include detailed verification data, while contributors remain responsible for reviewing and accepting the results. Current limitations include shared-utility refactors, vision-language models, quantized uploads, and thinking-specific tests.
Inside VAKRA: Reasoning, Tool Use, and Failure Modes of Agents
VAKRA is an executable, tool-grounded benchmark for evaluating whether AI agents can complete enterprise-like workflows that combine API calls, document retrieval, and natural-language constraints. It provides more than 8,000 locally hosted APIs backed by real databases across 62 domains, plus domain-aligned documents, and uses full execution traces to assess multi-step behavior. The four capability groups cover API chaining, dashboard-API tool selection, multi-hop dashboard reasoning, and multi-hop multi-source reasoning with multi-turn dialogue and optional tool-use policies; tasks can require three to seven reasoning steps, while some segments specify one to twelve tool calls or up to five logical hops. Reported analyses find that models perform best on single-hop questions, degrade as API hops increase, and often violate policies or fail to retrieve enough information when constraints restrict the relevant source, exposing a gap between isolated tool competence and end-to-end reliability.
Meet HoloTab by HCompany. Your AI browser companion.
HoloTab is a Chrome extension that brings HCompany’s computer-use AI into the browser. It navigates websites, fills fields, and makes decisions from a user’s description, with vision models, action planning, and interface understanding operating underneath without setup or technical skills. The announcement follows the March 31st release of Holo3, described as the company’s most advanced computer-use model to date. For repetitive workflows, users can record their actions and narration in a tab; HoloTab turns the recording into a routine that can later be rerun or scheduled. The stated goal is to make computer-use AI accessible beyond teams with engineers who can connect such systems, and the extension is presented as free and available for everyone today.
Agentic reasoning in practice: Making sense of structured and unstructured data
Enterprise questions that connect structured records with unstructured text are difficult for single-turn retrieval systems to answer reliably. The post evaluates Databricks Agent Bricks Supervisor Agent (SA), a declarative agent builder built on the internal aroll framework, across STaRK’s Amazon, MAG, and Prime domains and six KARLBench tasks. SA decomposes questions, routes sub-questions to Genie or Knowledge Assistant, combines results, and can reformulate queries or use SQL JOINs when initial retrieval is incomplete. It outperforms rerun SoTA baselines by 4% on STaRK-Amazon, 21% on MAG, and 38% on Prime, while delivering a 23% improvement on FinanceBench and consistent gains across KARLBench; the authors attribute the results to multi-step tool selection, reflection, and precise instructions rather than custom code.
Agentic coding with ClickHouse. One person, one data stack, one full-stack application
A Solutions Architect at ClickHouse built ClickShop, a full-stack retail analytics platform, within a few days using Cursor to demonstrate production-level capabilities beyond static slides. The platform pairs ClickHouse for sub-second analytical queries across billions of rows with PostgreSQL for transactional workflows, using ClickPipes for real-time change data capture synchronization. LibreChat orchestrates 18 persona-tailored AI agents, while dual observability layers—ClickStack for OpenTelemetry infrastructure metrics and Langfuse for LLM prompt and cost tracking—store their telemetry directly in ClickHouse. Developing with an AI-assisted IDE was accelerated by open-source documentation and community skills, though it required continuous human review to handle unintended code refactoring and token consumption.
Managing context in long-run agentic applications
Long-running multi-agent systems struggle with context management because accumulating raw message histories degrades inference quality, increases latency, and exceeds context window limits. In a collaborative security investigation platform, passing unrestricted history can also introduce confirmation bias across specialized agents. To maintain coherence across unbounded rounds of investigation, the system eliminates raw message history carryover between invocations. Instead, it coordinates agents through three structured context channels: a Director's Journal for orchestration memory, a Critic's Review that scores findings to filter hallucinations, and a Critic's Timeline of validated chronological events. This architecture provides agents with tailored context without overwhelming their inference capacity.
Automating Receipt Collection: Apple Intelligence for On-Device Inference
Ramp built automatic receipt detection for its iOS app to reduce the effort of finding receipt photos while keeping private images on device. An initial design used PhotoKit, Vision OCR, background processing, local storage, and exact checks for merchant, amount, and date, but formatting differences, aliases, and OCR errors produced many false negatives. FoundationModels enabled a local structured-output LLM, yet a single prompt performed poorly in evals because the small 4096-token model hallucinated and took shortcuts. Splitting the merchant, date, and amount questions and requesting rationale improved evaluation results, but the alpha release reached 66% precision and about 18% recall. Version 3.0 combined one LLM merchant-name check with deterministic iOS 26 data detectors for dates and monetary values; it cut processing time, more than doubled recall, and raised precision to 87%.
Introducing clickhousectl: the CLI for ClickHouse local and cloud (beta)
ClickHouse released the beta version of clickhousectl, an official command-line interface designed to unify local operations and cloud infrastructure management for both humans and AI agents. The tool acts as a version manager that stores reusable binaries globally in ~/.clickhouse/versions/ while offering optional project directory scaffolding for SQL files. Developers and agents can run multiple isolated local server instances concurrently with automatic free-port detection and dedicated data directories. For ClickHouse Cloud, the CLI provides complete management coverage with structured JSON output, using read-only OAuth or scoped API key authentication to safeguard modifying operations. Additionally, the tool can install ClickHouse Agent Skills directly into supported coding agents to facilitate automated local and cloud interactions.
AI Agents Know About Supabase. They Don't Always Use It Right.
Supabase is releasing Supabase Agent Skills, an open-source instruction set for AI coding agents working with Supabase through Claude Code, Codex, GitHub Copilot, Cursor, and compatible Agent Skills implementations. The skill addresses recurring errors such as omitted RLS policies, hallucinated CLI commands, views missing security_invoker = true, and reliance on outdated training data by requiring documentation checks and embedding critical security guidance in SKILL.md. It also guides CLI and MCP workflows and recommends direct schema changes on local or staging databases, followed by database advisors and a migration after the schema stabilizes. Early evaluations across six scenarios found MCP + Skill scores of 67% for Claude Opus 4.6, 71% for Claude Sonnet 4.6, 88% for Codex GPT-5.4, and 71% for GPT-5.4 Mini, though the sample was small.
The three villains to agentic observability: retention, sampling and rollups
Observability practices increasingly depend on data sampling, short retention windows, and rollups to control storage costs. While human operators historically compensated for these compromises with intuition, AI agents lack implicit operational context and require full-fidelity telemetry to diagnose issues and justify conclusions. Head sampling drops unobserved spans early, tail sampling risks missing implicit non-error anomalies, and aggressive log expiration prevents long-term cyclical trend analysis. Addressing these constraints requires decoupling storage and compute via high-compression object storage and dedicated compute pools. Deploying real-time analytical databases such as ClickHouse enables high-concurrency, low-latency SQL querying, log clustering, and structured in-database summarization necessary for automated reasoning loops.
How Respan is scaling LLM observability with ClickHouse Cloud
Respan scaled its LLM gateway and observability platform from hundreds of daily requests to 50 million daily events, causing its original Django and Postgres architecture to stall under transaction contention at 50 to 100 requests per second. To overcome Postgres write-ahead logging bottlenecks, the team migrated the ingestion and analytics pipelines to ClickHouse Cloud. Engineers minimized row sizes by truncating large prompt text and writing structured metrics to MergeTree tables, then built incremental minute- and hour-level materialized views alongside scheduled refreshable views for user metadata. Distributed tracing spans are ingested asynchronously with analytical parent-child resolution, utilizing functions like argMax rather than the FINAL modifier to prevent costly table merges. This architectural shift allows Respan to achieve sub-second search across tens of millions of records and support customer dashboards with predictable query performance.
Never stop disrupting yourself; introducing the Fin API platform
Fin API platform launch makes the models powering its customer-agent system available through APIs, with contracts starting at $250k annually. It says Fin currently resolves over 2M customer issues weekly across brands and runs on models trained by its AI group. Last week’s Apex announcement is described as a specialized customer-service LLM that beat frontier models from Anthropic and OpenAI in six-month production tests on resolution rate, latency, hallucination rate, and cost. Customers can choose the Fin Agent Platform, Fin Agent API, or direct access to Apex and related models for bespoke, specialized agents. The rationale is customer demand and a broader expectation that software differentiation will shift from features and interfaces toward agents built on differentiated AI.
Welcome Gemma 4: Frontier multimodal intelligence on device
Google DeepMind’s Gemma 4 family is available on Hugging Face as Apache 2-licensed, multimodal models intended for use across agents, inference engines, fine-tuning libraries, and on-device deployments. The family accepts image and text inputs across five sizes, while E2B, E4B, and 12B Unified also support audio; context windows range from 128k to 256k tokens. Its design combines alternating local and global attention, dual RoPE configurations, Per-Layer Embeddings, shared KV caches, and model-specific vision and audio handling, including an encoder-free unified 12B variant. The post documents integrations for Transformers, llama.cpp, MLX, WebGPU, Rust, and other tools, alongside fine-tuning examples and demos. Reported instruction-tuned results include a 1452 estimated text-only LMArena score for 31B and benchmark results spanning reasoning, coding, vision, audio, and long-context tasks.
Rill and ClickHouse: real-time operational BI for a metered world
Granular operations across cloud computing, payment platforms, and artificial intelligence workloads produce high-volume event streams that traditional business intelligence tools struggle to unify and aggregate. Operational BI platform Rill addresses this challenge by combining declarative data loading through the dlt framework with ClickHouse's high-performance analytical aggregation engine. Data transformations, SQL-based measure expressions, and dashboard configurations are defined entirely as code using SQL and YAML. This declarative, metrics-first architecture allows developers to validate analytics locally, track changes in Git, and eliminate intermediate caching layers when querying live data. By querying ClickHouse directly and linking AI-generated insights back to underlying queries, the system provides traceable, real-time operational analytics for enterprise environments.
Supabase docs over SSH
Supabase is launching supabase.sh, a free public SSH server that gives AI coding agents bash access to Supabase documentation as a public experiment. Its virtual file system exposes every documentation page as a Markdown file under /supabase/docs, allowing agents to use familiar commands such as grep, find, ls, cat, head, and tail to browse, search, and skim current content. The service runs on just-bash, Vercel’s open-source Bash emulator written in TypeScript, inside a sandboxed JavaScript runtime over a virtual file system, so it does not execute an actual shell on the server. Project instructions can be added with ssh supabase.sh agents >> AGENTS.md or generated through ssh supabase.sh setup | claude; the post contrasts this exploratory workflow with the existing search_docs MCP tool and presents supabase.sh as an early experiment.
Agentic coding at ClickHouse
Industry perspectives on AI coding agents remain divided between dismissal and excessive automation claims. ClickHouse integrated agentic coding across internal workflows, progressing from simple boilerplate scripts to active backend C++ development. Engineers primarily use command-line agents such as Claude Code with Claude Opus 4.5 and Codex CLI alongside observability platforms like Langfuse and Librechat. While agents effectively investigate continuous integration logs, generate internal dashboards, and assist with small C++ tasks, they lack architectural competence and require rigorous code review. ClickHouse treats agents as pair programmers for contained tasks, retaining human ownership over product strategy, attention to detail, and core code quality.
Introducing MongoDB Agent Skills and Plugins for Coding Agents
Coding agents often default to relational patterns, resulting in over-normalized schemas, underused compound indexes, and misconfigured search indexes in production MongoDB environments. To address these pitfalls, MongoDB launched official MongoDB Agent Skills alongside integrated plugins for tools including Claude Code, Cursor, Gemini CLI, and VS Code. These skills provide agents with discoverable instructions, schema design heuristics, indexing strategies, and operational safeguards across the development lifecycle. The skills operate alongside the MongoDB MCP Server, which handles database connectivity, authentication, and permission-based governance. Teams can install these bundled extensions directly or manage them via the Vercel Skills CLI to embed MongoDB architectural standards into automated workflows.
Liberate your OpenClaw
Anthropic is limiting access to Claude models in open agent platforms for Pro/Max subscribers, so the post describes ways to keep OpenClaw, Pi, or Open Code agents running with open models from Hugging Face. It presents Hugging Face Inference Providers as the quickest hosted route: users create a token, run openclaw onboard --auth-choice huggingface-api-key, and select a model such as GLM-5, or set a repo_id in OpenClaw’s config. For fully local inference, it recommends installing llama.cpp, starting a server with Qwen3.5-35B-A3B-GGUF, and connecting OpenClaw to the local OpenAI-compatible endpoint with custom configuration; the example model is described as suitable for 32GB of RAM. The hosted option prioritizes speed and model availability, while local inference offers privacy, zero API costs, full control, and no rate limits.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.