# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [Any Apple update can break our app. Here's how we find out first.](https://yomu.fyi/post/any-apple-update-can-break-our-app-here-s-how-we-find-out-first.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Dan Mindru, Oleh Stasula
- Published: Jul 2, 2026

Usage is an Apple ecosystem resource monitor that relies on undocumented APIs vulnerable to silent data structure changes across OS updates. To maintain stability across iPhone, iPad, and Mac without degrading device performance, the development team uses Sentry across separate component projects in a unified workspace. Beyond capturing standard crashes, the team wraps Swift errors with unexpected raw data payloads to quickly diagnose undocumented API structure drift while scrubbing sensitive user values. They also employ Sentry Size Analysis as a regression gate to monitor release payload deltas, preventing unintended asset omissions or bloat from slipping through code reviews. This multi-layered monitoring setup enables the team to detect and resolve platform-induced breakages before most end users encounter them.


### [How do you turn AI coding chaos into a repeatable playbook?](https://yomu.fyi/post/how-do-you-turn-ai-coding-chaos-into-a-repeatable-playbook.md)
- Company: [Stack Overflow](https://yomu.fyi/company/stack-overflow.md)
- Author: Eira May
- Published: Jul 2, 2026

Software development dynamics are shifting as code generation costs decrease, requiring engineering organizations to adapt their leadership and processes. Snowflake addressed this transformation across its software lifecycle by first encouraging widespread, unrestricted experimentation with coding agents before formalizing successful practices. The engineering organization codified these practices into fourteen distinct AI design patterns, including strategies such as planning in markdown and isolating parallel agents. This structured approach, combined with dedicated focus weeks and a progress scale, allowed engineering teams to accelerate both inner and outer loops of development. Consequently, a team of three engineers leveraged coding agents to deliver a 40x improvement to Snowflake's query compiler while significantly shortening release validation times.


### [Mastering Agentic Techniques: AI Agent Reinforcement Learning](https://yomu.fyi/post/mastering-agentic-techniques-ai-agent-reinforcement-learning.md)
- Company: [NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Jul 1, 2026

Language model agents often fail across long-horizon workflows, repeat tool-call errors, and output invalid schemas when relying solely on prompting or retrieval. Reinforcement learning with verifiable rewards provides a practical training signal for domain-specific tasks by scoring model rollouts against deterministic verifiers such as schema checkers, unit tests, and execution sandboxes. Teams can implement group relative policy optimization to sample multiple trajectories per prompt and update policy weights based on relative performance within the group. Establishing environments with explicit harness, state, and verifier definitions allows agents to learn multi-step tool use, whereas static datasets fail to capture sequential dependencies. Inspecting checkpoint rollouts and testing against held-out tasks prevents reward hacking while continuously turning operational failures into verifiable benchmark environments.


### [Meta’s AI Storage Blueprint at Scale](https://yomu.fyi/post/meta-s-ai-storage-blueprint-at-scale.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: By Sidharth Bajaj, Venkatraghavan Srinivasan
- Published: Jul 1, 2026

Meta redesigned its BLOB-storage architecture on top of the foundational Tectonic block layer to eliminate GPU stalls and accelerate AI training workflows. The legacy storage system suffered from multi-layered metadata lookups, cross-region latency, and dataplane proxy bottlenecks that conflicted with the millisecond access requirements of flash-based AI clusters. To resolve these issues, the team collapsed disparate metadata into a single flat schema backed by ZippyDB for O(1) path lookups and introduced a fat client SDK capable of streaming data directly from storage servers. The updated stack also leverages spare GPU host memory as a distributed data cache alongside distributed read-plan caches to handle severe traffic spikes during checkpointing and model loading. These protocol and architectural changes deliver predictable latencies, reduce power consumption, and maintain high throughput during distributed training.


### [Reading the agent traces is how you make the call your eval can't](https://yomu.fyi/post/reading-the-agent-traces-is-how-you-make-the-call-your-eval-can-t.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Sergiy Dybskiy
- Published: Jul 1, 2026

Automated evaluations often miss unknown failure modes in AI agents because developers only write tests for anticipated errors. While building an itinerary agent for AI Engineer World's Fair, an open-weight model routed a speaker query to a tool returning only tracks, leading the model to hallucinate famous speakers and falsely claim the tool provided that data. Because a tool was executed, the trace appeared grounded until inspectable tool outputs revealed the discrepancy. To resolve this, the author retained the inexpensive model, corrected query routing, tightened system prompts against ungrounded entities, and added a custom entity-diffing eval in CI. Ultimately, directly reading agent telemetry traces alongside errors and logs is essential for diagnosing root causes and deciding trade-offs between model costs, prompt adjustments, and routing fixes.


### [Designing GPU-Accelerated Query Engines with NVIDIA GQE](https://yomu.fyi/post/designing-gpu-accelerated-query-engines-with-nvidia-gqe.md)
- Company: [NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Jun 30, 2026

GPU-accelerated query engines encounter performance constraints stemming from memory and I/O bandwidth limitations. NVIDIA introduces GQE, a reference architecture using NVIDIA cuDF, nvCOMP, and other CUDA-X libraries to accelerate SQL query execution across modern hardware like the NVIDIA GB200 NVL4. The architecture coordinates three distinct layers: a query layer consuming Substrait plans, a data layer managing in-memory table transfers, and an execution layer executing relational operator task graphs. GQE employs pipeline parallelism, partition pruning, hybrid compression, and batched asynchronous transfers to overlap data movement with GPU compute. On a non-standard 1 TB TPC-H benchmark, GQE on a single B200 GPU achieved a 7.5x total execution time speedup over dual-socket CPU configurations running DuckDB.


### [10 Years of Meta’s Commitment to Python](https://yomu.fyi/post/10-years-of-meta-s-commitment-to-python.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Published: Jun 30, 2026

Meta has marked ten consecutive years as a sponsor of the Python Software Foundation (PSF) to ensure the language remains robust and sustainable. Python represents the most widely used programming language across Meta's infrastructure, powering applications like Instagram and Threads as well as internal AI research. Beyond financial backing, Meta engineers actively contribute core language features, author Python Enhancement Proposals (PEPs), and build open-source tools such as the Pyrefly type checker. Company sponsorship directly funds the PSF's Developer-in-Residence program, core infrastructure, and security enhancements for the Python Package Index (PyPI). Continued support of the PSF functions as a strategic investment in maintaining the stability of the broader open-source ecosystem.


### [Optimizing a Neural Reconstruction Pipeline Using NVIDIA Nsight Developer Tools](https://yomu.fyi/post/optimizing-a-neural-reconstruction-pipeline-using-nvidia-nsight-develo.md)
- Company: [NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jun 30, 2026

NVIDIA Omniverse NuRec reconstructs dynamic 3D environments from camera and lidar sensor data for autonomous vehicle simulation workflows, but high computational demands historically caused long reconstruction times. Engineers used NVIDIA Nsight Systems and Nsight Compute to systematically profile the PyTorch-based training pipeline and optimize its underlying CUDA kernels. Nsight Systems revealed GPU underutilization caused by numerous small kernels and blocking synchronization calls, leading to a fused interpolate kernel that accelerated execution from 4.184 milliseconds to 83.81 microseconds. Profiling with Nsight Compute uncovered resource mismatch in the renderBackward kernel, prompting the team to split it into dedicated camera and lidar implementations with tuned register and shared memory allocations. These adjustments raised kernel occupancy from approximately 15% to between 30% and 50% while decreasing the longest lidar kernel runtime from 31 milliseconds to 18 milliseconds.


### [Why intent prediction needs more than an LLM](https://yomu.fyi/post/why-intent-prediction-needs-more-than-an-llm.md)
- Company: [Stack Overflow](https://yomu.fyi/company/stack-overflow.md)
- Author: Phoebe Sajor
- Published: Jun 30, 2026

Traditional large language models rely on next-token prediction trained over broad text sequences, excelling at conversational synthesis and code generation while struggling with forecasting future user behavior. Predicting intent inherently requires modeling decision-making under conditions of uncertainty, a capability not directly produced by general text pretraining. Dedicated behavioral foundation models address this limitation by training representations directly on proprietary, sensitive behavioral data, such as anonymous browsing sessions and product identifiers. By learning base representations of behavior, these models generalize to unseen predictive tasks, such as optimizing ad campaigns for products absent from the original training dataset. Implementing these behavioral models also necessitates privacy-centric machine learning techniques, ranging from differential privacy concepts to homomorphic machine learning.


### [How to Govern Autonomous Agents in Enterprise AI Factories](https://yomu.fyi/post/how-to-govern-autonomous-agents-in-enterprise-ai-factories.md)
- Company: [NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Jun 29, 2026

Autonomous AI agents require governed environments to safely inspect code, execute tests, query enterprise systems, and handle sensitive business data. The NVIDIA Secure Agent Workspace Reference Design shifts agent execution away from user laptops to dedicated, single-user virtual machines managed on Red Hat OpenShift Virtualization or Microsoft Azure. Perimeter defenses enforce enterprise single sign-on access broker connections, default-deny network egress rules, and human approval gates for critical state-changing actions. Inside the workspace virtual machine, active sandboxing through NVIDIA OpenShell evaluates signed security policies before each tool invocation while credential proxies prevent direct access to raw secrets. GitOps workflows continuously reconcile infrastructure profiles and network configurations while all operational and security telemetry streams to centralized SIEM platforms in OCSF format.


### [GenPage: Towards End-to-End Generative Homepage Construction at Netflix](https://yomu.fyi/post/genpage-towards-end-to-end-generative-homepage-construction-at-netflix.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jun 29, 2026

Netflix traditionally constructs its structured two-dimensional homepage through a complex, multi-stage recommender pipeline that separates candidate generation and ranking across rows and entities. To simplify this architecture and optimize directly for whole-page user satisfaction, Netflix developed GenPage, an end-to-end generative transformer model that autoregressively builds the entire homepage from raw tokenized context. The system relies on a domain-specific tokenizer to compress engagement history, context injection for cold start, hybrid row decoding to minimize decoding steps, and reinforcement learning post-training. In online A/B testing against the production baseline, GenPage delivered statistically significant gains in core user engagement metrics while cutting end-to-end serving latency by 20 percent. Offline evaluations further showed that enriching context prompts improved recommendation quality more effectively than increasing model capacity in the current operating regime.


### [Next.js already traces your requests. Here's how to export them with OpenTelemetry.](https://yomu.fyi/post/next-js-already-traces-your-requests-here-s-how-to-export-them-with-op.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Kyle Tryon
- Published: Jun 29, 2026

Next.js provides out-of-the-box tracing for incoming requests, fetch calls, middleware, and server-side rendering, but traces remain invisible without a configured exporter. Developers can use the @vercel/otel package inside an instrumentation file to initialize the OpenTelemetry SDK and transmit traces to any OTLP-compatible destination. While Next.js automatically creates spans for standard request lifecycles, developers can define custom active spans in the Node runtime to capture specific domain operations and contextual attributes. Exporting traces to backends like Sentry requires configuring standard OTLP endpoint and authentication environment variables. Choosing between direct OTLP export via @vercel/otel and the dedicated Sentry SDK depends on requirements, as direct export lacks browser tracing, error monitoring, and Edge custom spans.


### [Creating the NVIDIA Nemotron 3 Ultra NVFP4 Checkpoint with NVIDIA Model Optimizer](https://yomu.fyi/post/creating-the-nvidia-nemotron-3-ultra-nvfp4-checkpoint-with-nvidia-mode.md)
- Company: [NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Jun 26, 2026

Moving large model weights efficiently requires quantization, but mapping weights to NVFP4's eight positive values risks severe precision loss or outlier-induced underflow. To address this, NVIDIA quantized the 550B Nemotron 3 Ultra model using NVIDIA Model Optimizer with a multi-precision strategy rather than quantizing all layers uniformly. The recipe applies four-over-six adaptive block scaling to Mixture of Experts routed weights, while assigning FP8 to shared experts and retaining BF16 for sensitive attention linears and embeddings. This process reduced the checkpoint size from 1,121 GB down to 352.3 GB and enabled single-checkpoint execution across Hopper using W4A16 and Blackwell using native W4A4. As a result, the model achieves up to 5.9x higher inference throughput than the GLM-5.1 754B FP4 model on decode-heavy workloads while matching BF16 benchmark accuracy.


### [Paging Charity! How can engineering leaders avoid becoming Bond villains?](https://yomu.fyi/post/paging-charity-how-can-engineering-leaders-avoid-becoming-bond-villain.md)
- Company: [Stack Overflow](https://yomu.fyi/company/stack-overflow.md)
- Author: Charity Majors
- Published: Jun 26, 2026

Engineering leadership advice often prioritizes personal empathy over commercial competence, which Charity Majors argues is a misdiagnosis of organizational failure. Because over 90% of venture-backed startups fail, tech culture frequently venerates ruthless founders who succeed commercially. New engineering managers frequently focus solely on subjective employee comfort, mistaking business instability or misaligned stakeholder priorities for micromanagement. To champion humane values sustainably, engineering leaders across all tiers must first master business operations, clarify product strategy, and deliver business wins. Advancing to the director level requires moving beyond generic cheerleading, building high-performing teams oriented around continuous improvement, and demonstrating systems thinking alongside business acumen.


### [Streamlining Resource Binding with End-to-End Support for Vulkan Descriptor Heaps](https://yomu.fyi/post/streamlining-resource-binding-with-end-to-end-support-for-vulkan-descr.md)
- Company: [NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jun 25, 2026

Vulkan introduces the VK\_EXT\_descriptor\_heap extension to refactor GPU resource binding, replacing set-based layouts and pool allocations with application-managed descriptor memory heaps. This approach aligns Vulkan closer to Direct3D 12 and benefits workloads like dynamic texture indexing and complex ray tracing shaders. Developers manage memory directly, binding a single resource heap and a separate sampler heap globally to avoid high rebinding overhead. Shaders can access descriptors either through mapping structures using constant offsets or push constants, or directly via untyped pointers enabled by VK\_KHR\_shader\_untyped\_pointers. Support is integrated into NVIDIA drivers 610, Vulkan Headers 1.4.340, and Nsight Graphics 2026.2 for frame capture and inspection.


### [Q&A: How KRAFTON Built PUBG Ally, a Co-Playable Character Powered by NVIDIA ACE](https://yomu.fyi/post/q-a-how-krafton-built-pubg-ally-a-co-playable-character-powered-by-nvi.md)
- Company: [NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Jun 25, 2026

KRAFTON developed PUBG Ally, an on-device AI co-playable character for PUBG: BATTLEGROUNDS, using NVIDIA ACE tooling and models. Conventional non-playable characters rely on fixed dialogue, but PUBG Ally processes live game states and player voice input in real time. The interactive pipeline combines automatic speech recognition, a quantized Mistral-NeMo-Minitron-2B small language model, and a custom text-to-speech model running locally on client GPUs with at least 8GB of VRAM. To minimize latency, the architecture separates reactive gameplay into a behavior-tree System 1 layer while routing strategic decisions and speech generation to the language model. KRAFTON tuned the non-deterministic system across multi-layer evaluations and large-scale playtests with over a thousand players.


### [How we used DSPy to turn AI evaluations into better responses in Dash chat](https://yomu.fyi/post/how-we-used-dspy-to-turn-ai-evaluations-into-better-responses-in-dash.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon
- Published: Jun 25, 2026

Dropbox improved its Dash chat agent by establishing an automated optimization loop powered by DSPy and LLM-as-judge evaluations. Engineers first calibrated their LLM judges against human-annotated interaction traces, then used those judges to systematically optimize the agent's system prompts via offline counterfactual replay. This automated workflow reduced incomplete responses by 26% while decreasing overall token consumption.


### [Achieving Near-Linear Training Scalability for Pinterest’s Foundation Models](https://yomu.fyi/post/achieving-near-linear-training-scalability-for-pinterest-s-foundation.md)
- Company: [Pinterest](https://yomu.fyi/company/pinterest.md)
- Author: Pinterest Engineering
- Published: Jun 25, 2026

Pinterest’s foundation ranking models store approximately 99% of their parameters in embedding tables, causing cross-node NCCL all-to-all communication to severely bottleneck multi-node distributed training. Initial multi-node scaling stalled at 1.21x on four nodes even after adopting AWS Elastic Fabric Adapter for OS-bypass networking. To resolve this, engineers profiled GPU activity with PyTorch Profiler and systematically addressed communication bottlenecks across multiple layers. The team implemented FP8 quantized communications using FBGEMM, balanced table sharding across GPUs, halved embedding dimensions while doubling row counts, and adopted a 2D parallel communication topology alongside PyTorch Distributed Checkpoint. These combined interventions achieved near-linear scaling factors of 3.9x on four nodes and 7.5x on eight nodes across 64 GPUs.


### [Automated Schema Evolution in Pinterest’s Next-Generation DB Ingestion Framework](https://yomu.fyi/post/automated-schema-evolution-in-pinterest-s-next-generation-db-ingestion.md)
- Company: [Pinterest](https://yomu.fyi/company/pinterest.md)
- Author: Pinterest Engineering
- Published: Jun 24, 2026

Pinterest's CDC ingestion platform relies on a multi-stage pipeline built with Kafka, Flink, Spark, and Iceberg, where upstream schema updates risk disrupting data transformations and offline storage. To prevent pipeline failures and schema drift, Pinterest introduced an automated schema evolution framework based on a staged convergence model. Upstream changes are detected via push-based DDL messages and daily pull-based validation checks. The workflow automatically restricts automated updates to additive changes and numeric precision widening, updating Iceberg table definitions and regenerating Flink and Spark code through auditable pull requests. By decoupling schema divergence, code deployment, and data convergence, the system maintains continuous ingestion availability while working toward eventual schema consistency across all storage layers.


### [Accelerating BEV Pooling on NVIDIA GPUs for Physical AI Applications](https://yomu.fyi/post/accelerating-bev-pooling-on-nvidia-gpus-for-physical-ai-applications.md)
- Company: [NVIDIA Developer Blog](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: John Yang
- Published: Jun 24, 2026

Bird's-eye-view (BEV) pooling frequently becomes an inference latency bottleneck in spatial AI pipelines due to irregular memory access patterns and scatter-reduce operations. BEVPoolV3 addresses this by tailoring kernel execution to the GPU memory regime, specifically distinguishing between DRAM-bound and L2-cache-resident working sets. The implementation reduces redundant scatter traffic via depth-outer traversal, precomputes indices to eliminate runtime integer division, minimizes duplicate depth loads, and adopts interval-owned writes to bypass atomic operations. On a canonical 49 MB nuScenes working set, BEVPoolV3 accelerates TensorRT plugin execution from 274.0 µs down to 16.4 µs on an NVIDIA RTX PRO 6000 Blackwell Max-Q GPU. Furthermore, profiling with Nsight Compute reveals that L2-resident scatter-reduce operators achieve optimal efficiency with FP8, avoiding the ALU instruction overhead of NVFP4 decoding.


[Newer posts](https://yomu.fyi/page/9.md) · [Older posts](https://yomu.fyi/page/11.md)