Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,668 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Introducing Cloudflare Basin: an open, serverless data platform, now generally available
Cloudflare announced the general availability of Basin, a serverless data analytics platform built on Apache Iceberg and R2 Object Storage. Basin consists of three integrated products: Pipelines for ingesting and transforming events via SQL, Catalog for managing Iceberg metadata and automatic table maintenance, and SQL for distributed querying. The platform emphasizes speed with instant resource creation and query execution, scalability through metadata compaction and statistics-driven query planning, and openness by supporting any Iceberg-compatible engine like DuckDB and Snowflake. Basin offers cost efficiency with zero egress fees and usage-based pricing, charging only when data is ingested, processed, or queried. Early adopters have used Basin for use cases including real-time e-commerce optimization, billing metrics storage, and infrastructure telemetry analysis.
Graph-centric agentic intelligence
Graphs encode relationships and provide scaffolding for causal inference and compositional reasoning, which is essential for AI agents reasoning over complex networks. The post traces the evolution of graph intelligence from passive topology models to active agentic reasoning, culminating in a three-pillar approach for root cause analysis. The approach combines a continuously synchronized network digital twin graph with a cascaded pipeline of graph algorithms—decomposition, clustering, and centrality ranking—that narrow the search space. An agentic orchestration layer selects and composes these algorithms adaptively based on failure characteristics, performing complexity triage and consulting an incident knowledge base. The system achieved root cause analysis in minutes on commercial networks, demonstrated with NTT DOCOMO at MWC.
A New Agentic Experience: JetBrains Air in IDEs – EAP Now Open
JetBrains launched the Early Access Program for Air, an agentic experience in its IDEs available as a plugin or in 2026.3 EAP builds. Air orchestrates multiple AI agents in parallel, moving beyond the traditional AI chat model that JetBrains found insufficient for concurrent tasks. It acts as a conduit for existing agents like Codex, GitHub Copilot, Junie, and Claude, requiring no JetBrains AI subscription. The agentic-first UX organizes work into sessions with cost visibility, temporary worktrees, and local or cloud runs, with free Junie Lite for everyday tasks. Air ships with no agents installed, detects agents on the machine, and sends data directly to the user’s provider, while AI Assistant remains available as a separate plugin.
Modal Clusters are generally available
Modal announced general availability of Modal Clusters, a serverless multi-node compute primitive for AI workloads that provides up to 6.4 Tbps InfiniBand RDMA per node with PyTorch and NCCL support. A new gang scheduler places N nodes simultaneously by observing the fleet, assigning all pending clusters, and provisioning capacity when needed. RDMA support is fully automated behind a single rdma=True flag, with Modal contributing RDMA support to the gVisor container runtime and upstreaming the changes. Customers like Decagon, 1x, and Runway use Clusters for trillion-parameter fine-tuning, pre-training home robot world models, and multi-node video inference respectively. Clusters are available to all workspaces today through the @modal.clustered decorator, with per-second billing and no reservation required.
How LinkedIn extended ClickHouse from distributed tracing to metric discovery and analytics
LinkedIn’s observability team first deployed ClickHouse for distributed tracing, where even at 1% head sampling the system generates 0.8 trillion spans and 200 TB of uncompressed data per day across three data centers. They then extended ClickHouse to consolidate a legacy metric metadata system built on RRDtool semantics, which had grown into three separate services that had reached their scaling limits. The new single-shard ClickHouse index serves over 150,000 queries per minute with an average latency of 68 milliseconds across 13 billion-plus metrics with 30% daily turnover. Migrating off the legacy stack reduced memory use to roughly one-fifth and compute to about two-thirds, with enough headroom to double the metric volume. By exposing individual dimensions as columns, the index enables engineers to replace regex-heavy queries with efficient native ones, supporting the larger migration away from RRD semantics.
RustRover vs VS Code + rust-analyzer: What Changes in Your Rust Workflow
This post compares RustRover and VS Code with the rust-analyzer extension for Rust development, detailing their architectural differences. VS Code, a general-purpose editor, relies on the rust-analyzer language server to provide completion, diagnostics, inlay hints, navigation, quick-fixes, and refactoring. RustRover is a dedicated professional IDE using JetBrains’ own analysis engine that builds a model of crates, dependencies, targets, and code structure from the Cargo workspace, with built-in debugging, testing, and version control. Key differences include RustRover’s integrated debugger versus VS Code’s need for an extension like CodeLLDB, and RustRover’s project-wide refactoring with affected-usage review. AI assistant capabilities also differ, with RustRover offering agent-driven tasks through Air and JetBrains AI while VS Code supports inline suggestions and agent providers like Copilot. The post offers a five-step migration guide for VS Code users and concludes the choice depends on whether one prefers a modular editor or an integrated IDE experience.
Barclays scales Claude to upgrade operations and improve client experience
Barclays, a British universal bank, is expanding its collaboration with Anthropic to integrate Claude across global operations, aiming to accelerate software development, modernize legacy systems, and improve efficiency. By end of 2026, Barclays expects Claude Code adoption to reach 50% of its developer population, rising to a majority of software engineers in 2027. A live example is the Colleague Knowledge Assistant, powered by Claude via a retrieval-augmented generation architecture since 2025, adopted by over 16,000 employees and handling over one million searches. In Global Markets, Claude models classify, enrich, and route approximately 120,000 client emails daily, reducing manual handling and helping staff prioritize requests. Barclays applies robust governance, security controls, and human oversight to ensure responsible AI deployment.
IP Functions are Generally Available, bringing high-performance network analytics to the Lakehouse
The announcement introduces the general availability of IP Functions, a feature that brings high-performance network analytics to the Lakehouse by enabling SQL-based processing of IP addresses. The post explains that IP data is warehousing data, as firewalls, load balancers, VPNs, CDN edges, DNS resolvers, Kubernetes clusters, and application servers emit streams of records keyed on IP addresses, generating tens of billions of events daily for large enterprises. These streams support valuable analytics such as threat detection, fraud investigation, and network governance. The post notes that network analytics used to be painful because IP addresses were deceptively hard to handle in SQL, with no regex support. The feature aims to simplify this by providing native IP functions for the Lakehouse.
ShopGym: Realistic, reproducible sandboxes for shopping agents (2026) - Shopify
ShopGym converts live storefronts into self-contained, resettable sandbox shops for training and evaluating shopping agents, addressing challenges from changing products and bot detection. Its ShopArena layer separates exploration from generation using anonymized specifications and coding agents with execution-verification loops that repair failures without regenerating the whole shop. ShopGuru then synthesizes benchmark tasks across seven skill categories using rule-based and LLM generators grounded in each shop’s catalog, navigation, and policies. Validation over 224 tasks across six shops, three synthetic and three real, showed synthetic shops retain key structural properties and agent performance positively correlates with live storefronts when tested with BrowserGym and an internal system. The framework aims to provide reproducible environments for agents to fail and learn before interacting with live stores, with plans to expand to more retail domains and task types.
Your customers don't want an AI friend. They want an answer.
A Twilio survey of 660 business leaders and consumers reveals a gap: 98% of organizations train AI to act more human, yet customers rank issue resolution, speed, accuracy, and availability above personality. Only 54% of consumers want human-like AI, versus 73% of brands. Behaviors customers call empathetic—acknowledging the specific situation, offering multiple solutions, using conversational language, offering compensation—depend on personalization and context, not tone. 51% of consumers find AI robotic, and 39% of businesses cite weak emotional understanding as a top dissatisfaction driver. The report recommends prioritizing context and personalization over personality, using tools like Twilio Conversation Memory and Conversation Orchestrator to make agents useful before charming.
Prioritize your messages at scale with the Twilio Traffic Optimization Engine
Twilio’s Traffic Optimization Engine is a new suite for global messaging deliverability and prioritization, tackling carrier network strain from soaring volumes like the 6.9 billion messages during Cyber Week 2025. The suite automates months of manual planning, working across networks, senders, geographies, and use cases to deliver high-priority messages on time. Real-time Routing uses patented algorithms to optimize traffic across 4,800+ carrier connections every 75 seconds, while Market Throughput manages rate limits at the country and sender-type level to prevent throttling. Traffic Shaping creates dynamic lanes to prioritize one-time passwords over promotional messages, and Multitenancy fairly distributes throughput among ISV teams. Insights provides real-time analytics on deliverability and capacity allocation, and the overall system uses traffic shaping and priority queueing based on carrier, sender, and message type capabilities.
VM Sandboxes: Full computers for agents
Modal has launched VM Sandboxes in general availability, providing agents with full Linux VMs accessible via a single flag. The VMs support Docker stacks, local databases, dev servers, and other real-machine features that agents increasingly require, as demonstrated by Ramp’s internal agent driving the entire software development lifecycle. Modal built a custom runtime on the Rust-based Cloud Hypervisor, layering innovations in host filesystems, lazy image loading, memory bursting, and snapshotting to preserve the same APIs and sub-second cold starts as their existing gVisor Sandboxes. Early customers like Linear, Legora, and Snorkel have launched over 20 million VMs, using them for coding sessions, legal agent evaluations, and agent simulations that require multiple containers and real-world environments. Modal recommends using VM Sandboxes when workloads need Docker, FUSE filesystems, or niche kernel features, while gVisor remains the default for compatible workloads.
Runtime Roundup: VM Sandboxes, Multi-node clusters, and more
Modal announced several new features at its inaugural Runtime conference. VM Sandboxes provide agents with full Linux computer access, supporting Docker stacks, databases, graphical environments, and kernel experimentation, already used at Linear, Legora, and Snorkel for coding agents and evals. Sandbox Sidecars run alongside main sandboxes with isolation but share a host, allowing agent code in the main sandbox while trusted operations run in a sidecar the agent cannot access, avoiding network round trips in execution-heavy loops. Modal Clusters, now generally available via @modal.clustered, enable multi-node jobs for training or serving trillion-parameter models, powering post-training at Decagon and real-time inference at Runway. Sticky Sessions maintain client state by routing to the same container using a session token, critical for voice model serving, while Endpoint Candidates allow mirroring production traffic to compare quality, latency, throughput, and cost before promoting.
Sidecars: A low-latency trust boundary for Sandboxes
Modal introduces Sidecars, isolated containers that run alongside a main Sandbox on the same host, establishing a trust boundary between trusted and untrusted code. Sidecars enable communication across trust boundaries at least 3x faster than using separate Sandboxes, reducing latency for operation-heavy agent workloads. The feature addresses the security risk of running an agent’s harness and tool calls together, which creates the “lethal trifecta” of private data, untrusted content, and external communication. Existing solutions either move the harness to a remote service (adding network latency) or rely on egress controls that lack flexibility. Sidecars allow trusted services such as credential injection, proxy traffic, and harness logic to run locally alongside the Sandbox, with independent network policies and dynamic creation via the Modal SDK.
RLTL;DR: Self-Improvement by Internalizing Self-Generated Feedback
Standard reinforcement learning with verifiable rewards (RLVR) fails when tasks are so difficult that agents have no chance of success and no teacher models exist. RLTL;DR addresses this by having the policy write its own TL;DR insight after each failed attempt, conditioning the next rollout on all previous insights. Backpropagation on the in-context insights internalizes a direct task-to-insight mapping. On challenging tool-calling and coding datasets filtered to Pass@128=0, standard GRPO training stays flat at 0-1% Pass@1, while RLTL;DR achieves 14-31% Pass@1 with insights in context during training and 12-13% without at eval time. Reducing to SFTL;DR, training on only 4k (task, insight) tuples without rollouts, recovers almost full performance of RLTL;DR and classical SFT, demonstrating a promising compacted training paradigm.
How Much of a Harness Does a Strong Agent Need for Autonomous ML Engineering?
Recent autonomous machine learning engineering agents increasingly rely on elaborate machinery such as multi-agent orchestrators and dedicated retrieval subagents, motivated by progress stagnation over long-horizon cycles and limited Large Language Model primitives. This paper finds that under an equal time budget and the same frontier LLM backbone, open-source state-of-the-art harnesses provide no advantages over a single session of a minimal-harness coding agent baseline that uses direct read, write, and bash primitives. The authors argue via large-scale systematic ablation studies that the machinery layers become redundant in the coding agent setting. The backbone, not the harness complexity, is the primary driver for performance. Consequently, the effort spent elaborating hand-crafted harnesses around strong models yields poor returns for current MLE benchmarks.
Read Restrictions and Catalog Labels: Unifying governance across engines and catalogs
The Apache Iceberg community has adopted two additions to the Iceberg REST Catalog — read restrictions and catalog labels — each addressing distinct governance challenges. Read restrictions standardize delegated enforcement: a catalog evaluates policy for a principal and returns column-projection actions and row-filter expressions from a bounded vocabulary for a trusted engine to enforce. Catalog labels enable portable governance by allowing catalogs to exchange opaque key-value metadata at table and column level, so a consuming catalog applies native policies without per-user calls to the source. Read restrictions suit engine-to-catalog access where the source catalog trusts the engine and policies fit the standard vocabulary, whereas catalog labels fit federation between heterogeneous catalogs needing scalable governance and independent audit trails. These building blocks, along with centralized enforcement via cross-engine ABAC, advance the vision of unified governance across the open lakehouse.
How to scale agentic applications without creating AI sprawl
As agentic applications move from answering questions to taking actions, enterprises risk AI sprawl from duplicated integrations, inconsistent policies, and fragmented context. Databricks asserts scaling requires three pillars: choice across models and tools, context grounded in governed enterprise data and business semantics, and control through consistent permissions, policies, observability, and cost management. Its platform provides Agent Bricks for unified agent fleet building and governance, Omnigent as a common layer above agent harnesses, and Unity Gateway as a centralized control plane for models, agents, and tools. Context is shared via Unity Catalog for data and AI asset governance and Genie Ontology for common business concepts, letting teams reuse governed context across applications. Control uses Unity Gateway for access policies, budgets, and traces and Databricks Sandbox for isolated execution, with MLflow enabling evaluation and workflow inspection to maintain reliability as agents scale.
A practical guide to cost optimization with Lakebase Postgres
Lakebase uses separated storage and compute to reduce costs: branches share storage, autoscaling matches compute to demand, and scale to zero suspends compute after inactivity. Read replicas and high availability add compute without duplicating storage. For synced tables from Lakehouse, sync only the active working set via materialized views and choose the appropriate sync mode—snapshot for volatile data, triggered for incremental, continuous for real-time—and binpack tables into one pipeline to share compute. Right-size initial compute by focusing on the working set, and set PITR retention and snapshot schedules based on actual recovery needs. Snapshot storage is priced at $0.090/GB-month and PITR storage at $0.200/GB-month, both cheaper than regular branch storage. Compute and storage usage can be tracked through system.billing.usage, with separate SKUs for compute, branch storage, PITR, and snapshots.
Introducing SCIM Provisioning in ClickHouse Cloud
ClickHouse Cloud introduces SCIM 2.0 provisioning, making the identity provider the single source of truth for user membership and role assignments. Previously, SAML SSO handled authentication but left user invites and role updates as manual tasks, creating security risks. SCIM provisioning automatically syncs user create, update, and delete operations from the IdP using standard SCIM 2.0 endpoints, maps IdP groups to custom ClickHouse roles via stable external IDs, supports up to two API keys for rotation, and provides audit logging for all provisioning actions. The feature requires an active SAML connection, is limited to verified domain SSO users, excludes system roles like Admin, and revokes all keys if SAML is disabled. It is generally available for Enterprise and BYOC organizations, validated with Okta and Microsoft Entra ID, and eliminates manual invite and deprovisioning tasks.
Top topics
- Architecture834
- Performance815
- LLMs659
- Machine Learning621
- Open Source563
- AI Agents553
- AI469
- Developer Experience448
Weekly digest
The best engineering reads, once a week.