Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,737 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Mastering Agentic Techniques: AI Agent Reinforcement Learning
Language model agents often fail across long-horizon workflows, repeat tool-call errors, and output invalid schemas when relying solely on prompting or retrieval. Reinforcement learning with verifiable rewards provides a practical training signal for domain-specific tasks by scoring model rollouts against deterministic verifiers such as schema checkers, unit tests, and execution sandboxes. Teams can implement group relative policy optimization to sample multiple trajectories per prompt and update policy weights based on relative performance within the group. Establishing environments with explicit harness, state, and verifier definitions allows agents to learn multi-step tool use, whereas static datasets fail to capture sequential dependencies. Inspecting checkpoint rollouts and testing against held-out tasks prevents reward hacking while continuously turning operational failures into verifiable benchmark environments.
How to measure the customer experience as AI scales
CSAT captures fewer than 10% of conversations, and its responses skew toward delighted or furious customers. Most customers say nothing, leaving teams to assess customer experience from an unrepresentative sample that compresses product, policy, and service issues into one score. As AI handles more conversations end-to-end, the post argues for scoring every AI and human interaction on service quality, resolution, customer effort, and related attributes rather than relying on survey responses alone. Fin’s CX Score assigns each interaction a 1–5 score and surfaces reasons behind it, providing roughly five times more coverage than CSAT for most teams. Fin sets targets from CX Score attributes, operational correlations, historical performance, modelling, and team ambition, then uses recurring patterns to route fixes to support, product, or workflow owners.
Meta’s AI Storage Blueprint at Scale
Meta describes how its BLOB-storage architecture evolved to address AI workloads' storage bottlenecks, which can stall GPUs and slow research iteration as datasets and GPU deployments expand. Legacy requests traversed multiple stateful metadata layers, sometimes across regions, producing latencies of hundreds of milliseconds even when flash made millisecond access possible. The redesign consolidates metadata into a flat schema backed by ZippyDB for O(1) path resolution, removes the dataplane proxy in favor of a fat client SDK, and deploys regional BLOB-storage stacks beside GPUs. For data loading, Meta combines host memory and flash with regional flash and global HDD-backed storage as tiered caches, using ordinary and deep prefetch, on-demand hydration, and TTL or LRU eviction policies. Production rollout saw rapid adoption, while both data-loading paradigms remain supported.
amazon.science
How Amazon tracks carbon intensity across its operations
Amazon measures carbon intensity across its diverse business operations to track progress toward its goal of reaching net-zero carbon by 2040. For retail operations, the company monitors carbon emissions per unit shipped, achieving a 39 percent reduction by the end of 2025 compared to 2019 baselines. This per-unit decrease stems from targeted investments in electric delivery vehicles, carbon-free energy, smarter package routing, lightweight packaging, low-carbon fuels, and alternative transport modes like rail. To improve industry comparability and transparency, the company updated its economic carbon intensity indicator in 2025 from emissions per dollar of gross merchandise sales to emissions per dollar of revenue. Carbon intensity is also evaluated regionally and nationally to isolate geographic variations and implement localized decarbonization interventions across supply chain networks.
How we scale PgBouncer in ClickHouse Managed Postgres
PgBouncer is single-threaded, so a single process can bottleneck connection pooling on multi-vCPU machines even when Postgres has capacity. ClickHouse Managed Postgres runs a fleet of PgBouncer processes sized to available cores, sharing one port through so_reuseport; peering forwards query-cancellation requests to the process that owns the session, while transaction pooling and per-process connection-budget splits limit Postgres connections. In tests on identical AWS EC2 c7i.4xlarge pooler instances, a 16-process fleet reached roughly 336,000 transactions per second at 256 clients, compared with 76,893 for one process, while using about 49% versus 8% in-guest CPU. The fleet's advantage emerged under concurrency, whereas the single process was slightly faster at eight clients, and the described setup ships by default with every ClickHouse Managed Postgres server.
Forecasting at the speed of modern retail
Retail and CPG forecasting now spans hundreds of thousands, sometimes far more, time series across fragmented channels, promotions, and short-lived products, making legacy methods and manual exception management difficult. Multi-model forecasting addresses this complexity by evaluating a range of techniques against actual data and selecting the best-performing model for each series, but enterprise-scale experiments require scarce forecasting and distributed-systems expertise. Released in 2024, Databricks’ open-source Many Model Forecasting (MMF) integrates more than 35 statistical, deep-learning, and foundation time-series models and runs on distributed Databricks compute. MMF Agent, built on Genie Code, guides users through data quality, series classification, compute configuration, forecasting, post-processing, and model selection, while Unity Catalog helps it use organizational data context. The workflow is intended to reduce setup from days to hours, improve targeting and accuracy, and make rigorous forecasting more accessible while remaining customizable for technical teams.
How Artemis Security runs 69x faster detection queries with ClickHouse Cloud
Artemis Security uses ClickHouse Cloud on AWS for real-time threat detection across many terabytes of compressed logs, tens of billions of daily rows, and queries over trillions of rows. To reduce repeated scans, its engineers coalesce hundreds of detection-rule predicates into one query, tagging matching rows by rule ID; batches of more than 100 rules now run in under a second, with a measured 69x speedup. A Claude Code skill called /poke connects ClickHouse system tables, CloudWatch, Datadog, application data, and source code to investigate infrastructure issues through natural-language prompts. For investigative queries over semi-structured JSON, Artemis extracts frequently queried fields into a parallel materialized table and uses a hybrid two-step lookup, producing 30–60x faster queries and up to 400x lower CPU consumption.
Celebrating the Winners of the 2026 Built-On Databricks Startup Challenge - Cloned
The 2026 Built-On Databricks Startup Challenge recognized early-stage startups building B2B applications on the Databricks platform, with winners announced at the Data + AI Summit in San Francisco on June 16. VisionHeight won the grand prize for an agentic threat-intelligence platform that maps adversary infrastructure across the Internet while it is being built; Linkup placed second with a production-grade Web Search API delivering results in about two seconds, and Intelo placed third with an agentic workforce for retail merchandising and planning. Clarecast, Gemini Sports, and LakeFusion received honorable mentions for predictive intelligence, football squad planning, and an AI-native data foundation combining MDM, PIM, and LakeGraph. Judges assessed market potential, founding-team caliber, and innovative use of Databricks, while the startup-program offer provides qualifying startups up to $200,000 in credits across Databricks and Neon.
Reading the agent traces is how you make the call your eval can't
Automated evaluations often miss unknown failure modes in AI agents because developers only write tests for anticipated errors. While building an itinerary agent for AI Engineer World's Fair, an open-weight model routed a speaker query to a tool returning only tracks, leading the model to hallucinate famous speakers and falsely claim the tool provided that data. Because a tool was executed, the trace appeared grounded until inspectable tool outputs revealed the discrepancy. To resolve this, the author retained the inexpensive model, corrected query routing, tightened system prompts against ungrounded entities, and added a custom entity-diffing eval in CI. Ultimately, directly reading agent telemetry traces alongside errors and logs is essential for diagnosing root causes and deciding trade-offs between model costs, prompt adjustments, and routing fixes.
Multi-token Residual Prediction
Multi-Token Residual Prediction (MRP) adapts multi-token prediction for diffusion language models by training a small module to predict the residual between adjacent denoising steps rather than the full next-step distribution. This residual target is easier to learn, allowing a tiny 3-layer transformer to accurately predict corrections across multiple steps. In the static regime, MRP enables lossless speculative decoding with up to 1.56× throughput in SGLang, or direct decoding with a small quality cost. In the dynamic regime, MRP recovers up to +16 accuracy points lost by aggressive low-threshold decoding through a remasking mechanism. The module is trained on a frozen backbone and composes cleanly with existing inference methods.
Announcing our $800M Series C to accelerate the shift to open-source AI
Together AI announces an $800M Series C to accelerate its open-source AI platform as production demand for inference grows. The company says the round includes $800 million from investors including Aramco Ventures, NVIDIA, Vista Equity, General Catalyst, and others, plus commitments for more than 500 MW of compute capacity to be capitalized independently by new investors. Together argues that closed frontier-model economics become unsustainable as agents and workflows scale, while open-weights models such as DeepSeek, Nemotron, MiniMax, Kimi, and GLM can provide equal or better performance at lower cost. Its full-stack approach spans models, kernels, compilers, inference systems, training infrastructure, hardware utilization, and software, with cited work including FlashAttention-4, Together Megakernel, together.compile, and expanded post-training APIs for tool calling, reasoning, and vision-language models.
Build agentic full-stack apps with Genkit
Genkit Agents is an open-source, full-stack foundation for conversational AI and agentic applications, supporting TypeScript, Go, Dart, and Python. The Agents API packages message history, tool loops, streaming, persistence, and frontend communication behind a shared interface: agents can run in process or behind HTTP, and clients use chat() through remoteAgent(). Developers can choose client-managed state or session-store-backed snapshots, resume by session ID, branch from snapshot ID, stream text, custom state, and artifacts, and pause tools for human approval. The framework also supports detached long-running work, specialist delegation, persistence choices including in-memory, file, Firestore, and custom stores, plus an Agent Runner in the Developer UI. The Agents API is in preview for TypeScript and Go, with breaking changes possible in minor releases.
ML Development in VS Code with Google Cloud Power: Workbench Extension Now Available
Google Cloud has launched the Google Cloud Workbench Notebooks extension for VS Code, allowing developers to use managed cloud notebook environments from a local IDE. Workbench, described as a managed Jupyter platform optimized for data science, can provide access to Google Cloud's AI-optimized infrastructure and high-performance compute when local machines need more capacity. The workflow connects through VS Code's Extensions view, the Jupyter extension, and a notebook's Select Kernel menu, where users choose Google Cloud, Workbench, authenticate, select a project, and choose an active instance. The extension runs notebooks on Workbench instances, preserves local settings and extensions, is fully open-sourced, and is available through the VS Code Marketplace, with GitHub contributions invited.
Why we built ADK 2.0
ADK 2.0 addresses the structural challenge of moving AI agents from prototypes into production, where infinite loops, hallucinated bypasses, and failures without clean exceptions make model-led orchestration unreliable. Because language models are slow, costly, and variable for routing, scheduling, and error handling, the release separates execution routing from language processing and reserves LLMs for ambiguous or cognitive steps. Building on ADK v1’s support for Python, Java, Go, TypeScript, and Kotlin, it adds a structured workflow runtime and task-collaboration model, with Workflows available in Python since March and newly launched for Go. Deterministic graphs combine tool calls, Human-in-the-Loop steps, and specialized agents with explicit branching and error handling. Dynamic workflows extend this model with native Python and asyncio control flows for retries, branching, and reusable sub-workflows.
From monolith to Lakebase to LTAP: rethinking the database from storage up
The post examines why monolithic OLTP databases become fragile and difficult to scale when compute, the write-ahead log (WAL), and data files share one machine. Using Postgres as its primary example, it contrasts local WAL durability and physical cloning with Lakebase, whose stateless Postgres compute externalizes those components into SafeKeeper and PageServer services. SafeKeeper replicates log records across a quorum through Paxos-based network replication, while PageServer applies the WAL and materializes data in cloud object storage. The resulting separation is presented as a way to improve durability, independently scale reads and writes, isolate workloads, and support high availability and branching without physical database clones. LTAP extends the design by making transactional tables directly queryable for analytics from a single governed copy, avoiding CDC or mirroring and allowing transactional and analytical engines to scale independently.
Designing GPU-Accelerated Query Engines with NVIDIA GQE
GPU-accelerated query engines encounter performance constraints stemming from memory and I/O bandwidth limitations. NVIDIA introduces GQE, a reference architecture using NVIDIA cuDF, nvCOMP, and other CUDA-X libraries to accelerate SQL query execution across modern hardware like the NVIDIA GB200 NVL4. The architecture coordinates three distinct layers: a query layer consuming Substrait plans, a data layer managing in-memory table transfers, and an execution layer executing relational operator task graphs. GQE employs pipeline parallelism, partition pruning, hybrid compression, and batched asynchronous transfers to overlap data movement with GPU compute. On a non-standard 1 TB TPC-H benchmark, GQE on a single B200 GPU achieved a 7.5x total execution time speedup over dual-socket CPU configurations running DuckDB.
10 Years of Meta’s Commitment to Python
Meta has marked ten consecutive years as a sponsor of the Python Software Foundation (PSF) to ensure the language remains robust and sustainable. Python represents the most widely used programming language across Meta's infrastructure, powering applications like Instagram and Threads as well as internal AI research. Beyond financial backing, Meta engineers actively contribute core language features, author Python Enhancement Proposals (PEPs), and build open-source tools such as the Pyrefly type checker. Company sponsorship directly funds the PSF's Developer-in-Residence program, core infrastructure, and security enhancements for the Python Package Index (PyPI). Continued support of the PSF functions as a strategic investment in maintaining the stability of the broader open-source ecosystem.
Optimizing a Neural Reconstruction Pipeline Using NVIDIA Nsight Developer Tools
NVIDIA Omniverse NuRec reconstructs dynamic 3D environments from camera and lidar sensor data for autonomous vehicle simulation workflows, but high computational demands historically caused long reconstruction times. Engineers used NVIDIA Nsight Systems and Nsight Compute to systematically profile the PyTorch-based training pipeline and optimize its underlying CUDA kernels. Nsight Systems revealed GPU underutilization caused by numerous small kernels and blocking synchronization calls, leading to a fused interpolate kernel that accelerated execution from 4.184 milliseconds to 83.81 microseconds. Profiling with Nsight Compute uncovered resource mismatch in the renderBackward kernel, prompting the team to split it into dedicated camera and lidar implementations with tuned register and shared memory allocations. These adjustments raised kernel occupancy from approximately 15% to between 30% and 50% while decreasing the longest lidar kernel runtime from 31 milliseconds to 18 milliseconds.
A Quadrillion Rows across three Clouds: scaling LogHouse
LogHouse now stores 431 PiB of uncompressed data across 1.59 quadrillion rows, spans more than 30 regions across three cloud providers, and serves typical queries in a few hundred milliseconds. The platform grew 23x from 19 PiB in two years, while peak ingestion reached 80 GiB/s and 190 million rows per second. Its write architecture uses local geosharding, multiple cells in busy regions, and ClickHouse Async Inserts to combine small incoming writes into larger parts without overwhelming merges. On the read side, a single Distributed table hides the cell topology, while unavailable shards remain visible by default rather than producing silently incomplete results. Current work includes reducing index and metadata memory, improving async-insert durability, collecting system data from object storage, and expanding OpenTelemetry traces and metrics.
Agentic Risk Operations
Ramp describes an architecture for scaling risk operations as payment volume, payment rails, partners, and country coverage expand, rather than letting headcount grow with the business. Agents handle universal intake, gather business context, classify requests, and route work, while machine-learning models trained on millions of historical data points and approved policies make autonomous risk decisions. Operators can modify agent skills and tool configurations without engineering involvement, deploy changes in shadow mode, and rely on asynchronous execution, provider failover, and centralized observability. In payment-risk workflows, structured operator feedback produced a benchmark of more than 1,000 operations covering tool-call trajectories, operator alignment, and downstream outcomes, enabling separate evaluation of agents and policies. Ramp rolls agents out beside operators, then scales operation types and dollar volume using performance thresholds and capped exposure budgets.
Top topics
- Architecture834
- Performance817
- LLMs673
- Machine Learning622
- Open Source568
- AI Agents564
- AI483
- Developer Experience448
Weekly digest
The best engineering reads, once a week.