Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Ultra-Fast Anomaly Detection using Apache Spark Real-Time Mode
The post presents a reusable real-time guardrail pattern for flagging suspicious Ethereum blockchain transactions and routing them for downstream action. Its rules identify impossible blocks where gas_used exceeds gas_limit and scan extra_data for email addresses, JWT tokens, or AWS access-key patterns, producing ALLOW or QUARANTINE decisions with reasons. The implementation uses Apache Spark Structured Streaming Real-Time Mode, whose continuous data flow, pipeline scheduling, streaming shuffle, pre-allocated execution pipelines, and asynchronous checkpointing target millisecond latency without a separate streaming engine. On a four-worker DBR 16.4 LTS cluster, the stateless Kafka-to-Kafka test processed about 23 million records at 69,713 rows per second, with P95 below 0.5 milliseconds and P99 at 1 millisecond. The post notes that RTM with a Kafka sink provides at-least-once delivery and that more complex stateful workloads may incur higher latency.
Modernizing the Meta Ads Service With an Open-Source Kernel Scheduler
Meta’s ads serving fleet processes more than 5 million requests per second, so millisecond-scale latency changes can affect ads relevance and advertiser ROI. During a move from kernel 6.4 with CFS to kernel 6.9, the team observed a latency regression associated with EEVDF and used sched_ext, an upstream BPF-based framework, to encode ads-specific scheduling priorities. The policy soft-partitions CPUs between latency-critical request-path threads and less-sensitive work, adjusts pool sizes with load-based heuristics, and runs as a user-space binary that loads BPF, enabling policy changes without rebuilding or reinstalling the kernel. Initial results were a 28% reduction in ads retrieval p99 latency, 3.28 megawatts of fleet power savings, and a 1.1% increase in weighted ads ranked; two later user-space updates delivered an additional 60% p99 reduction and an 18% decrease in critical-path timeout errors, with iterations shipping in days rather than months.
Claude for Creative Work
Claude for Creative Work describes a set of connectors and workflows intended to help creative professionals expand ideation, reduce repetitive work, and take on larger-scale projects. The connectors connect Claude with tools including Ableton, Adobe Creative Cloud, Affinity by Canva, Autodesk Fusion, Blender, Resolume, SketchUp, and Splice, supporting documentation access, asset production, 3D modeling, live visual control, and sample search. The described uses also include tutoring, Claude Code scripts and plugins, cross-application pipeline work, Claude Design exploration with export to Canva, and batch processing. Blender has an officially available MCP connector that can analyze and debug scenes, batch-apply object changes, and add tools through Blender’s Python API; because it uses MCP, it is accessible to other LLMs. The post also describes access for students and faculty in three art and design programs, whose feedback will inform understanding of creative practitioners’ needs.
Anthropic Sydney office
Anthropic has officially opened a Sydney office and appointed Theo Hourmouzis as General Manager for Australia and New Zealand. Hourmouzis brings more than 20 years of technology leadership experience across Asia Pacific and most recently served as Snowflake’s Senior Vice President for Australia, New Zealand and ASEAN. He will lead the growing local team, shape a regional strategy for customers, and bring Claude into important work. The team will deepen relationships with enterprise customers and AI for Science research partners while delivering on commitments in a memorandum of understanding with the Australian government. Local partnerships include Canva, Xero, and YMCA South Australia, where custom Claude AI skills turn operational data into actionable insights, reduce branded content production from hours to minutes, and bring previously outsourced technical work in-house.
Extreme Event Likelihoods with Guided Generative Models
Estimating the likelihood of extreme, high-impact events using brute-force Monte Carlo simulations requires excessive model iterations for expensive physics-based climate models. While guided diffusion models steer generation toward rare events like tropical cyclones, oversampling distorts the true probability distribution. To address this, researchers guide the diffusion-based climate emulator NVIDIA cBottle toward specific events and compute log-odds ratios between guided and unguided distributions. Reweighting the guided samples via these odds ratios enables importance sampling under the original climate distribution. Implemented in NVIDIA Earth2Studio, this technique reduced standard error by 25% compared to simple Monte Carlo sampling when evaluating tropical cyclone states.
How Insider One uses ClickHouse Cloud to power real-time customer engagement at scale
Insider One’s Unified Customer Database (UCD) unifies event data, user attributes, identity resolution, and profiles across channels to support real-time segmentation, personalization, and analytics. The platform initially ran ClickHouse in a self-managed AWS deployment, but growth brought larger datasets, more concurrent queries, and increasing work tuning clusters and balancing performance against cost. In a proof of concept, the team migrated an anonymized dataset from one of its largest customers to ClickHouse Cloud and replayed UI segmentation, audience-count, and scheduled-campaign workloads. Average query latency fell 50–60% without significant changes, reaching 70–75% for the largest customers, while latency percentiles became more predictable; the managed service also reduced operational burden and gave the team more capacity for product development and its AI-oriented roadmap.
Practical Guide To Python App Hosting
Python app hosting is presented as both an infrastructure choice and, for data-intensive or AI applications, a data-architecture choice. The guide defines a hosting stack comprising application code, runtime, dependency manager, application server and reverse proxy, and contrasts it with traditional web hosting. It compares shared hosting, VPSs and cloud VMs, PaaS, container platforms and serverless functions by control, operational effort, scalability and workload fit. It notes that long-running data and AI tasks such as model training, vector indexes, cached datasets, ETL jobs and lengthy inference pipelines can exceed practical serverless limits. The conclusion is that public APIs and lightweight web apps allow broad platform choice, while apps accessing governed data, model endpoints or AI agents should be evaluated by where data lives and how access is governed, with Databricks Apps presented as an option when data already resides in a lakehouse.
How to Evaluate General-Purpose Robot Policies for Real-World Deployment
Evaluating robotics foundation models accurately in simulation remains challenging due to visual domain overlap, benchmark saturation, diagnostic gaps, and small sample sizes. RoboLab addresses these deficiencies by providing a robot-agnostic simulation platform that supports rapid task creation and diagnostic evaluation suites. Tasks in RoboLab can be compiled across varying robot embodiments and target visual, procedural, and relational competencies. Statistical analysis using the Clopper-Pearson method reveals that narrowing the confidence interval around a 90 percent success rate from ten to two percentage points demands increasing rollouts from 70 to 1,030. Developed by NVIDIA Research, RoboLab powers the open-source Isaac Lab-Arena simulation framework to establish scalable evaluation for real-world robotic policies.
The agentic marketing stack starts with the data layer
The piece argues that agentic marketing depends on modernizing the data and infrastructure foundation first, rather than starting with isolated AI use cases. At Acxiom, moving core products and client solutions from on-premises Hadoop to Databricks reportedly improved workload runtimes by 80 to 90 percent, reducing jobs that took 50+ hours or as long as 90+ hours to 2–3 hours. The migration also reduced manual pipeline and infrastructure work, freeing engineering capacity for products and client outcomes. Those capabilities now support agentic audience planning, media buying, campaign activation, performance analytics, code generation, testing, and ad variation, with a goal of connecting the full marketing value chain. Because workflows handle PII, generated content passes through legal approval, defined controls, and human oversight; Acxiom is also embedding its data in client platforms and privacy-safe clean rooms to make decisions more transparent and native to customer environments.
Reducing High-Bandwidth Memory Bottlenecks in JAX-Based LLM Training with Host Offloading
Large language model training workloads frequently face GPU high-bandwidth memory (HBM) capacity constraints before compute limits are reached. To alleviate this memory pressure in JAX training workloads, host offloading transfers selected forward-pass activations to pinned host memory over high-speed NVLink-C2C interconnects and reloads them during the backward pass instead of recomputing them. Benchmarks executed on 128 GPUs within NVIDIA GB200 NVL72 systems using MaxText demonstrated significant improvements. For DeepSeek-V3 671B, host offloading combined with the Latency Hiding Scheduler and pipelined offloading reached 908.2 TFLOPs/s/device, surpassing activation rematerialization throughput by 57% and unlocking larger batch configurations. Similarly, QKV activation offloading on Llama 3.1 405B achieved a 2.9% throughput increase when transfer latency was hidden behind computation.
amazon.science
Amazon and University of Michigan give robots a sense of touch
Robotic manipulation in contact-rich settings requires accurate tactile force perception, but existing physics simulators either run too slowly for reinforcement learning or oversimplify shear forces. HydroShear addresses this tactile reality gap by incorporating path-dependent force tracking into hydroelastic contact models. By remembering object motion history across soft elastomer sensors, the GPU-parallelizable simulator computes full 3-D contact deformations, friction, and slipping dynamics. Reinforcement learning policies trained entirely in HydroShear transferred directly to a real Franka robot equipped with GelSight Mini sensors without fine-tuning. Across four challenging manipulation tasks, HydroShear achieved a 93 percent average success rate, outperforming baseline models TacSL and FOTS.
Introducing Feature Views
Feature Views are presented as a framework for creating managed feature pipelines in Databricks, addressing duplicated logic, training/serving skew, backfills, online-store plumbing, and governance for real-time ML. A Feature View defines the source, entity, time-series column, and computation once; Databricks uses it to generate point-in-time-accurate historical data, then materializes pipelines for batch or streaming production use. Registered in Unity Catalog, materialized features receive lineage and access controls, while MLflow records model dependencies and Model Serving retrieves required features without custom lookup code. For Kafka-sourced streaming features, Spark Realtime Mode, Lakebase, and Model Serving support rolling-window updates with reported end-to-end p99 latency of 200ms from event to online availability. The announcement says Feature Views are in Public Preview and can be developed through the Feature Engineering Client SDK and Genie Code, with streaming materialization requiring an Enterprise-tier workspace in a region supporting Lakebase.
Kernel Fusion in NVIDIA CUDA: Optimizing Memory Traffic and Launch Overhead
GPU compute often outpaces memory bandwidth, causing separate kernels to suffer from intermediate global memory round-trips and launch overhead. Kernel fusion resolves this by executing multiple operations within a single kernel to keep intermediate data inside registers. Using the sum(abs(x)) operation as a benchmark on an NVIDIA GeForce RTX 4090, a naive two-kernel setup moved 3 GB of global memory over 3.51 ms. In contrast, manual CUDA C++ fusion, PyTorch compiler fusion via torch.compile, and explicit Python fusion using cuda.compute all achieved a 3x speedup. Across all three approaches, global memory transfers dropped to 1 GB by computing absolute values inline during reduction.
Inspiration – Tools & craft – Notion
Notion’s Inspiration page gathers perspectives on the future of work from people and teams creating companies, communities, and creative practices. Its TechFirst Block pairs Ivan Zhao’s claim that AI has arrived as “infinite minds” with stories about organizations using Notion to change how work gets done. Featured examples include Brainlabs’ company-wide AI operating layer, Kim Shui’s Notion-organized New York Fashion Week preparations, and Her Tech Circle’s reconstruction of a community around a shared workspace. Additional profiles cover Thred’s effort to make offices less lonely, Sarah Jutras’s work teaching self-employment through Freelance for Life, and how leading AI companies implement AI tools effectively; the page also identifies contributors such as founders, marketers, designers, and community leaders.
AI Model Co-Design: Hardware-Friendly LLM Design
Efficient large language model serving requires co-designing architectures to balance datacenter token throughput and user interactivity within hardware constraints. Roofline analysis shows that linear layers become memory-bound when hidden or intermediate projection dimensions are small, leaving arithmetic hardware underutilized even at large token counts. For prefill workloads, Chunked Pipeline Parallelism reduces first-token latency on long contexts when models use regular, repeatable layer patterns that split into balanced stages. For latency-critical decoding, decoupling attention and feed-forward network parallelization enables independent optimization, applying expert parallelism for feed-forward networks and Helix parallelism to shard the key-value cache across sequences. Following architectural design rules—such as dimension alignment, width prioritization, and low-precision NVFP4 compatibility—allows models to achieve higher utilization on modern accelerator hardware.
Better tools made Copilot code review worse. Here’s how we actually improved it.
When migrating GitHub Copilot code review from custom navigation tools to shared Unix-style CLI utilities—grep, glob, and view—benchmarks revealed higher review costs and fewer caught issues. Rather than an issue with the underlying tools, trace analysis showed the agent used general-purpose coding assistant instructions that triggered expansive repository browsing loops. In response, the team rewrote the tool guidance to enforce a review-specific workflow anchored to pull request diffs. The updated instructions direct the agent to narrow candidate call sites and files using batched discovery before reading minimal line ranges with view. This workflow tuning reduced average review costs by approximately 20% in production while maintaining review quality.
ClickHouse on Docker Hardened Images
ClickHouse is now available as Docker Hardened Images (DHI), stripped-down builds for the ClickHouse server, ClickHouse Keeper, Kubernetes Operator and Helm chart, and metrics exporter. The standard image’s Ubuntu 22.04 base includes unused packages such as Perl, wget, and apt, whose known CVEs can block enterprise deployments even though ClickHouse does not use them. DHI replaces that base with a minimal image that has no package manager or network tools, runs as non-root, provides SLSA Level 3 provenance, and is maintained for CVE patching; Docker Scout reported 0 medium and 14 low findings for DHI, versus 8 medium and 11 low across 111 packages in the standard image. Migration preserves volumes and configuration, while development tooling remains available through a dev variant or temporary docker debug attachment.
One Driver, One Format, Every Language: ADBC
ClickHouse announces an official ADBC driver, a modern Arrow-native alternative to ODBC and JDBC for analytics and AI applications. It provides end-to-end columnar movement: query results leave ClickHouse as Apache Arrow and arrive in applications without a row-oriented conversion or staging step. A single Rust-built implementation connects through ClickHouse's HTTP interface, supports TLS, and serves languages and tools with ADBC bindings, including Ruby, R, and C. Users can install precompiled, signed binaries through the ADBC Driver Foundry and connect with a clickhouse:// URI; dbt Fusion distributes the driver directly. The driver also supports streaming Arrow inserts, while schema discovery, improved bulk ingest, usage examples, and JSON support remain on the roadmap.
Accelerating End-to-End Co-Folding Performance with NVIDIA BioNeMo Agent Toolkit
Biomolecular structure prediction and co-folding pipelines encounter latency and memory ceilings during sequence alignment and multi-residue inference. NVIDIA addresses these constraints through the BioNeMo Agent Toolkit, which connects AI agents to accelerated biological computing components. The pipeline offloads multiple sequence alignment to MMseqs2-GPU, while cuEquivariance and the OpenFold3 NIM accelerate core geometric attention kernels to cut inference latency up to fourfold. To handle large molecular assemblies beyond single-GPU memory limits, the Fold-CP framework distributes context across multiple GPUs with O(N²/P) memory scaling. Together, these tools enable the prediction of complexes up to 32,000 tokens on 64 NVIDIA B300 GPUs, making large-scale virtual screening and ribosome-scale modeling computationally feasible.
The ambulatory intelligence gap
Health systems face an ambulatory intelligence gap: access, provider capacity, referral retention, panel management, and financial performance are interconnected, while information remains scattered across disconnected systems. Health Catalyst’s Ambulatory Intelligence combines AI with nearly two decades of healthcare improvement expertise, deploying directly in a customer’s Databricks workspace so sensitive data stays within the health system’s environment. It uses a medallion-based semantic layer, Unity Catalog for governance, Lakebase for low-latency serving, and Genie alongside dashboards to help leaders investigate why metrics change. The solution ships with prebuilt metrics across Access Optimization, Revenue Intelligence, Panel Management, and Referral Insights, plus cross-domain scorecards and configurable terminology and workflows. Reported outcomes from supported improvement work include increased revenue and encounters at Thibodaux Regional, 55,000 closed care gaps at INTEGRIS Health, and higher outpatient visits with fewer cancellations without reschedules at WakeMed; future plans include models based on prior outcomes and agentic capabilities.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.