---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [How I built agent-based security reviews on Databricks](https://yomu.fyi/post/how-i-built-agent-based-security-reviews-on-databricks.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Angel De Leon
- Published: Sep 24, 2026
- Type: Problem & solution

Security reviews at Databricks relied on automation but still consumed expert time on predictable work, leaving less room for genuinely novel, high-risk architectures. The author built an agent-based layer on Databricks using Unity Catalog for governed tables containing standards, request data, and decisions; Databricks-hosted foundation models (Claude Haiku, Sonnet, Opus) for classification and reasoning; and Lakeflow Jobs for orchestrating notebook-based workflows on serverless compute. A conversational intake app built with Databricks Apps turns plain-language descriptions into structured requests and provides preliminary risk indications before a formal request is created. Seven focused agents each have a bounded responsibility—intake, risk assessment, requirements, specialized reviews, validation, workflow, and learning—ensuring behavior remains inspectable and testable. Routine requests that once waited days in the queue can now be completed in minutes, and security engineers spend more time on designs requiring genuine expertise.


### [Hitting a billion tokens per minute on one GPU by combining a query planner and an inference engine](https://yomu.fyi/post/hitting-a-billion-tokens-per-minute-on-one-gpu-by-combining-a-query-pl.md)
- Company: [Modal](https://yomu.fyi/company/modal.md)
- Published: Sep 24, 2026
- Type: Problem & solution

Modal and CMU built Quail, a query-aware inference layer for AI-SQL that combines a query planner with an inference engine to process structured LLM transformations of database entries. On a multi-join query, Quail achieves over a billion tokens per minute per H100 GPU, over 10x faster than vLLM, costing under 6 cents per billion tokens on Modal. The gain comes from using the query plan to order requests for better KV cache management, including a revised Hydragen-style cascade attention and batching small requests to reduce host overhead. On a new AI-SQL benchmark, Quail runs 1.84x faster than vLLM, but falls behind on agent trace processing. The authors see this as part of a trend toward custom inference engines for specific workloads, with a reusable core like databases post-DataFusion, and note that much work was done by prompting coding agents.


### [How to Test and Fix Voice AI Agent Prompts with Twilio Conversation Relay and Cekura](https://yomu.fyi/post/how-to-test-and-fix-voice-ai-agent-prompts-with-twilio-conversation-re.md)
- Company: [Twilio](https://yomu.fyi/company/twilio.md)
- Author: Paul Kamp
- Published: Sep 24, 2026
- Type: Tutorial

This tutorial demonstrates building a voice AI agent with Twilio Conversation Relay, Node.js, and the OpenAI API, then testing and fixing its prompts using Cekura automated platform, a Twilio Startup Searchlight award winner. Conversation Relay manages real-time voice tasks like speech-to-text, text-to-speech, interruption detection, and turn management over WebSockets, while developers control business logic, tools, and AI integration. Cekura defines evaluation metrics, generates test conversations with realistic caller scenarios—interruptions, topic changes, and unexpected requests—and runs baseline tests to uncover prompt gaps. After analyzing failure reports, developers update the system prompt and re-run tests; fixing a fee-waiver bug raised the pass rate from below 70% to the 90-95% target in one refinement. The tutorial emphasizes that prompts are source code requiring version control and automated testing for reliable voice AI agent behavior.


### [Measuring the back/forward cache with Application Metrics](https://yomu.fyi/post/measuring-the-back-forward-cache-with-application-metrics.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Abdelrahman Awad
- Published: Sep 24, 2026
- Type: Announcement

The back/forward cache (bfcache) allows instant page restores by freezing snapshots, yet its hits register no events in traditional telemetry, creating a blind spot for teams. Sentry’s bfcacheMetricsIntegration fills this gap by modeling three browser-provided signals as application metrics: a navigation counter with hit/miss outcome, a per-reason counter for misses via the notRestoredReasons API, and a distribution of fallback reload duration. The integration is opt-in and launched in JavaScript SDKs v11; detailed miss reasons are currently Chromium-only due to browser API support. Users can query these metrics to break down bfcache health by route, release, or browser, enabling discovery of regressions caused by unload listeners, cache-control headers, or third-party scripts. This turns the bfcache from an invisible performance feature into a measurable, trackable signal that teams can monitor and improve.


### [A Practical Recipe for Semi-Supervised Federated ASR: Online Pseudo-Labels with Server Update Stabilization](https://yomu.fyi/post/a-practical-recipe-for-semi-supervised-federated-asr-online-pseudo-lab.md)
- Company: [Apple](https://yomu.fyi/company/apple.md)
- Published: Sep 24, 2026
- Type: Problem & solution

Semi-supervised federated learning (SSFL) for ASR uses a teacher to generate pseudo-labels on unlabeled client data while the server trains on a small labeled seed. The paper finds that pseudo-label errors compound across the output sequence and training rounds, causing divergence and a large gap to fully-supervised FL. Two coupled design axes close this gap: the teacher type (per-client online, global broadcast, or transitioning) and the anchor (server-side labeled-data updates that stabilize training). A per-client online teacher diverges on its own but, once stabilized, matches or beats the global teacher in-domain and under domain shift. The server must keep training on labeled data between rounds; otherwise the online teacher drifts, and this interleaving governs convergence more than the seed model does. The proposed guidelines improve over the strongest prior method on 9 of 11 pairs, by 20.8% in-domain and 10.0% cross-domain on average.


### [Turn your REST APIs into MCP tools with Google Cloud API Gateway](https://yomu.fyi/post/turn-your-rest-apis-into-mcp-tools-with-google-cloud-api-gateway.md)
- Company: [Google](https://yomu.fyi/company/google.md)
- Author: Sanjay Pujare, Paul Howell
- Published: Sep 24, 2026
- Type: Announcement

Most enterprise REST APIs are invisible to AI agents, forcing teams to run separate MCP servers that duplicate their existing gateway logic. Google Cloud API Gateway in Public Preview solves this by acting as a remote MCP server — annotate the OpenAPI 3.0.x/3.1.x spec with x-google-api-management.mcp and deploy it with no extra infrastructure. The gateway accepts MCP JSON-RPC requests on a single /mcp endpoint, transcodes tools/call into the corresponding REST call, runs it through existing policies, and returns the response as an MCP result. Authentication via JWT or API key works unchanged for REST operations; tools/list is unauthenticated by default and requires JWT in production for security. With no separate server to operate, teams in Public Preview can expose up to 1,000 tools per gateway that support existing authentication and quota, and the MCP server is automatically discoverable via API hub and Agent Registry.


### [Reproducing OLMo 3 7B Pre-training in MaxText: case study of large scale training on TPUs](https://yomu.fyi/post/reproducing-olmo-3-7b-pre-training-in-maxtext-case-study-of-large-scal.md)
- Company: [Google](https://yomu.fyi/company/google.md)
- Author: Gagik Amirkhanyan, Ran Ran, Aireen Mei, Matt Davidow, TPU Inference Software Engineering Team, Google Cloud, AI2 Team
- Published: Sep 24, 2026
- Type: Problem & solution

The post describes the reproduction of AI2’s OLMo 3 7B pre-training using MaxText on Google Cloud TPUs, covering both stages, matching the reference loss curve and held-out metrics. PyTorch weights were converted to JAX with logit parity, KL divergence 1.5e-3 and 98.75% top-1 agreement, and held-out evaluations caught a data loader bug that made incorrect gains appear. The run demonstrated reliability via exact checkpoint-resume with zero loss delta, resilience to a 75% capacity loss via resize with 1% throughput preservation, and seamless TPU generation change. Performance optimizations achieved 44.5% MFU on Ironwood, and head reshaping from 32 heads by 128 to 16 by 256 yielded 12.4% faster throughput at identical parameters and FLOPs. The post emphasizes that faithful reproduction requires verification on generalization metrics and provides instructions for readers to reproduce the run with open sourced code.


### [Compressing Streaming Neural Audio Encoders via Latent-Space Distillation](https://yomu.fyi/post/compressing-streaming-neural-audio-encoders-via-latent-space-distillat.md)
- Company: [Apple](https://yomu.fyi/company/apple.md)
- Published: Sep 24, 2026
- Type: Problem & solution

Apple’s on-device dictation tokenizer maps waveform to latent representations for a sparsely activated language model, and compressing it reduces memory and latency demands. The authors compress it via latent-space distillation, using the teacher’s pre-quantizer latent as the supervision target rather than tokens or output distributions. A student encoder is trained to regress the teacher’s per-frame latent with a squared-error objective, and an affine layer absorbs width mismatch. This recipe works for both token interfaces and for tokenizers pretrained alone or jointly with a language model, requiring no fine-tuning. At 2.8x compression, the student stays within 1.9% relative WER of its teacher on five of six pairs and outperforms an independently trained tokenizer of identical capacity by 3.9% relative.


### [Welcoming Gaby Garcia, our new Recruiter](https://yomu.fyi/post/welcoming-gaby-garcia-our-new-recruiter.md)
- Company: [Resend](https://yomu.fyi/company/resend.md)
- Published: Sep 24, 2026
- Type: Announcement

Resend announced the hiring of Gaby Garcia as a new full-cycle recruiter who combines a people-first approach with strategic sourcing and a passion for creating thoughtful, scalable recruiting processes. Garcia holds a background in Psychology, which she says brings curiosity and empathy to her work, and she continually looks for ways to improve how teams hire. She spent the last five years at Automattic hiring engineers and designers, making her well acquainted with developers and the DevTools space. Garcia expressed excitement about Resend’s focus on culture and growing together, stating she loves helping people find a place where they can do meaningful work and feel like they belong. She aims to help Resend grow while being part of that kind of team herself, bringing her expertise to the company’s hiring process.


### [Bringing Private Processing to Meta AI Glasses](https://yomu.fyi/post/bringing-private-processing-to-meta-ai-glasses.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: By Pritam Shah, Oskar Linde
- Published: Sep 24, 2026
- Type: Announcement

Meta AI glasses require cloud processing for advanced AI features like streaming transcription and long-term recall, but traditional cloud architectures expose data during use. Private Processing uses confidential virtual machines with Trusted Execution Environments (TEEs) on CPUs and GPUs to keep data encrypted even while being computed. Before sending context, glasses perform remote attestation, verifying the server’s TEE certificate against a public transparency ledger that records all deployed binary images. Non-targetable routing via blind-signed tokens and third-party OHTTP relays prevents tying requests to specific users, while encrypted storage ensures data is only accessible with user-provided keys. This infrastructure, already proven for WhatsApp and Meta AI, extends to glasses, with security audits by firms like NCC Group and a bug bounty program for independent research.


### [Introducing NV-Reason-CT Open 3D CT VLM for Radiologist Chain-of-Thought Reasoning](https://yomu.fyi/post/introducing-nv-reason-ct-open-3d-ct-vlm-for-radiologist-chain-of-thoug.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Sep 23, 2026
- Type: Announcement

NVIDIA released NV-Reason-CT, a vision language model for 3D CT analysis that processes volumes as true 3D input rather than independent 2D slices. The model generates structured diagnostic reports covering 30 chest and 29 abdominal abnormalities and emulates radiologist chain-of-thought reasoning with systematic anatomical reviews and uncertainty articulation. Its architecture pairs a Qwen3.5-4B language model with a full 3D ViT encoder that processes volumes resampled to 192³ voxels at 2mm isotropic resolution using 8x8x8 patch tokens and 3D MRoPE for spatial token relationships. Training uses supervised fine-tuning on roughly 550,000 QA examples from CT-RATE, NIH CT, and CancerVerse followed by Group Relative Policy Optimization that refines reasoning quality based on diagnostic accuracy. NV-Reason-CT is an open research foundation not a cleared clinical product and sits within NVIDIA’s Medical AI ecosystem alongside NV-Reason-CXR, NV-Segment-CTMR, and NV-Generate-CTMR.


### [Claude discovers a novel enzyme system](https://yomu.fyi/post/claude-discovers-a-novel-enzyme-system.md)
- Company: [Anthropic](https://yomu.fyi/company/anthropic.md)
- Published: Sep 23, 2026
- Type: Announcement

Anthropic launched a new life sciences research group to use AI agents for fundamental biology discovery. In one of their first programs, Claude agents autonomously analyzed over 200,000 reverse transcriptase sequences from a DNA database. After 21 hours and 210 million tokens, one agent spotted a repeating DNA pattern next to a reverse transcriptase gene, which it identified as a CRISPR-like system. The system, called ART (array-associated reverse transcriptases), consists of the RT, an accessory protein, and an array of non-coding DNA repeats that are expressed as distinct small RNAs. Its function remains unknown, but the pattern of repeats is only found together in other programmable DNA-cutting systems, suggesting ART may be a new tool for biotechnology.


### [Validate GPU Cluster Readiness Before AI Workloads Land](https://yomu.fyi/post/validate-gpu-cluster-readiness-before-ai-workloads-land.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Sep 23, 2026
- Type: Explainer

GPU clusters can pass standard health checks yet fail under real AI training workloads due to a single degraded GPU, network link, or misrouted traffic. NVIDIA Cluster Readiness Engine (NVCRE) is an open source Kubernetes controller that runs real distributed workloads across topology-aware node groups before production jobs land, measuring results and reporting which nodes failed each test. Its layered API uses Custom Resource Definitions for Certification, Workflow, and Job resources, attributing each failure to a specific node and category such as NCCL bandwidth or training goodput. The WorkloadRun API handles platform detection, NCCL settings, shared-memory volumes, and gang scheduling, so operators do not need to write those manifests by hand. NVCRE integrates with NVSentinel for passive telemetry and is available under Apache 2.0, with a roadmap for inference and automated lifecycle validation.


### [How to Use NVIDIA Warp and MjWarp to Accelerate Robotics Simulation and Learning Workflows](https://yomu.fyi/post/how-to-use-nvidia-warp-and-mjwarp-to-accelerate-robotics-simulation-an.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Johnny Nuñez Cano, Asier Arranz, Rishabh Chadha, Ben Oliveri
- Published: Sep 23, 2026
- Type: Tutorial

NVIDIA Warp is a Python framework for writing GPU-accelerated kernels that compiles statically typed Python to CUDA. MuJoCo Warp (MJWarp) implements MuJoCo physics on Warp, enabling batched simulation of hundreds to thousands of independent robot environments on NVIDIA GPUs. The core API transition is small: mjw.put\_model and mjw.put\_data replace mujoco.MjModel and mujoco.MjData, and mjw.step advances the entire batch. Proper throughput measurement requires GPU synchronization and warm-up to avoid timing queue latency instead of actual GPU work. The post demonstrates moving an SO-101 follower arm from a single CPU world to 2,048 parallel MJWarp environments, validating parity and measuring aggregate world-steps per second.


### [Rendering huge pull requests in the GitHub Copilot app](https://yomu.fyi/post/rendering-huge-pull-requests-in-the-github-copilot-app.md)
- Company: [Github](https://yomu.fyi/company/github.md)
- Author: Alberto Gimeno
- Published: Sep 23, 2026
- Type: Problem & solution

Rendering a pull request with over a million changed lines and hundreds of threaded comments requires a fundamentally different architecture than a code-only diff. The standard virtualized approach fails because comment heights depend on render-time factors like markdown wrapping, expandable sections, and reply composers. The team split the document’s height into two independent domains: deterministic code geometry and dynamic block geometry for comments. A single idle- and scroll-gated measurement pass, scoped to the viewport, corrects heights without causing reflows mid-scroll. The result is a pull request view where a million-line diff with hundreds of comments opens, scrolls, and behaves like a normally sized pull request.


### [Manage Kubernetes Node Fleets with NodeWright](https://yomu.fyi/post/manage-kubernetes-node-fleets-with-nodewright.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Sep 23, 2026
- Type: Problem & solution

Managing Kubernetes node fleets, especially GPU nodes, is difficult because host-level changes can disrupt long-running training jobs, and traditional tools are not cluster-aware. NodeWright, an open-source Kubernetes-native package manager from NVIDIA, solves this by declaratively configuring and updating node operating systems while respecting PodDisruptionBudgets and non-interruptible workload labels. The operator orchestrates a sequence of cordon, wait, drain, apply, interrupt, and uncordon on each node, using container image packages that carry modifications such as kernel parameters, security agents, and verification scripts. It supports progressive rollouts across a fleet via DeploymentPolicy compartments with configurable disruption budgets and strategies like batch, canary, and rolling. NodeWright integrates with NVIDIA AI Cluster Runtime for validated configurations and is part of the DSX OS open-source portfolio, available under Apache 2.0.


### [How Concurrence governs clinical AI at a trillion-token scale with Unity Gateway](https://yomu.fyi/post/how-concurrence-governs-clinical-ai-at-a-trillion-token-scale-with-uni.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Ali Khokhar, John Xing, Tony Shi, Kelly Albano
- Published: Sep 23, 2026
- Type: Explainer

Concurrence processes 100.8 billion input tokens and 11.2 million LLM calls every 30 days, at an annualized rate of 1.2 trillion tokens. To govern these workflows, the company consolidates on Databricks with Lakebase for operational state, Unity Catalog for governance, and Unity Gateway for centralized AI access. Patient data is recorded as immutable events in Delta tables, from which a current patient state—called the world model—is computed while preserving provenance. All agents are tested against simulated patients before deployment; simulation traffic is approximately seven times production traffic, using immutable event history to replay states. Concurrence routes all coding-agent traffic through Unity Gateway, attributing each request to the engineer, and enforces compliance-gated model routing where HIPAA requirements are met first.


### [Unity Catalog Pages: a governed home for your business knowledge in Genie Ontology](https://yomu.fyi/post/unity-catalog-pages-a-governed-home-for-your-business-knowledge-in-gen.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Tim Feng, Ashish Singh
- Published: Sep 23, 2026
- Type: Announcement

Business concepts like active customer or completed trip are often defined differently across teams, scattered in Slack threads and spreadsheets, causing AI agents to guess confidently when answering questions. Databricks introduces Unity Catalog Pages as a governed home within Genie Ontology where data stewards with Genie Code define the authoritative meaning of those terms and entities. Pages complement existing semantics — metric views how to calculate, domains where to look, and certification what to trust — by capturing what each concept means. Genie Code can bootstrap Pages by extracting definitions from Confluence, Slack, Google Docs, or GitHub via MCP-connected tools, turning weeks of work into minutes. When Genie One uses a Page to ground an answer, the Page is cited as a clickable source so users can inspect the definition and verify the reasoning.


### [How I massively improved my AI inference performance without buying new hardware](https://yomu.fyi/post/how-i-massively-improved-my-ai-inference-performance-without-buying-ne.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Boaz Ben Shabat
- Published: Sep 23, 2026
- Type: Problem & solution

Default inference configurations for large language models often leave significant performance unused, even on expensive hardware like 16 NVIDIA H200 GPUs across two nodes. The author tested this setup with vLLM and Red Hat AI Inference 3.4 on a CoreWeave Kubernetes cluster, using enterprise RAG workloads with 2,000-token inputs and 100 concurrent users. By systematically comparing aggregated and prefill-decode (PD) disaggregated architectures, and tuning tensor parallelism (TP) and PD split, latency was dramatically reduced. For Qwen3-32B, time-to-first-token (TTFT) on the best aggregated configuration (2xTP8) dropped from 995ms to 304ms, and with optimized routing to 287ms — a 71% improvement without hardware changes. For Llama-70B, TTFT fell from 1,292ms to 463ms, a 64% gain. The author also built ServeIt Studio, an open-source tool that automates the full optimization pipeline on Kubernetes.


### [Build OpenJDK container images locally using the standalone S2I tool](https://yomu.fyi/post/build-openjdk-container-images-locally-using-the-standalone-s2i-tool.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Francisco De Melo Junior
- Published: Sep 23, 2026
- Type: Tutorial

Red Hat provides the Red Hat build of OpenJDK for running Java applications on Red Hat OpenShift 4. Developers can use the standalone Source-to-Image (S2I) CLI to build and test OpenJDK container images locally before deploying to a cluster. S2I produces reproducible images by injecting source code into a builder image and running its assemble script. The toolkit supports flags such as –loglevel for detailed output, –environment-file to pass custom JVM settings like G1GC and MaxRAMPercentage, and –as-dockerfile to output a Dockerfile. After building, the image can be tested locally with docker run, then uploaded to the cluster for deployment.


[Newer posts](https://yomu.fyi/posts/page/34.md) · [Older posts](https://yomu.fyi/posts/page/36.md)