Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Stop Building MFA from Scratch (And What to Do Instead)
Building multi-factor authentication (MFA) in-house is a much larger project than teams expect, requiring authenticator app integration, QR code generation, secure backup recovery codes, and edge cases like lost phones. Auth0 positions its MFA offerings as a way to offload that work so developers can focus on core product features. The company’s paid plans include MFA directly, with Pro MFA covering standard authenticator apps in B2C and B2B Essentials plans. Enterprise MFA adds WebAuthn, push notifications, SMS, and voice, available as an add-on to B2B Essentials or included in Professional tiers. Auth0 argues that using its MFA helps teams pass compliance, unblock enterprise deals, and avoid weeks of building login infrastructure.
Running open-Jev in SQL on Databricks
System One decision models such as Jev produce well-calibrated decisions from discrete options and are fast and cheap for large-scale applications, with potential use cases ranging from analyzing customer support transcripts to applying complex decisions at scale. Open-weight versions like SemIf-OpenJev are available, and Databricks enables serving them on governed data using serverless GPUs via AI Runtime, which provides on-demand GPU compute without managing infrastructure. The workflow is packaged in an importable notebook that downloads the model files, logs and registers the model using Express Deployments, and creates a GPU Model Serving endpoint automatically. The endpoint can be called from SQL via ai_query, which supports arbitrary custom model APIs, allowing structured request formats directly from SQL to return selected classifications and option probabilities as structured columns. The example classifies hotel reviews as good or bad, and users can import the notebook, select Serverless GPU, and run all to get started.
Meet (again) Max Schoening, Notion's new CTO
Notion appointed Max Schoening as CTO, reflecting its ethos of merging technology with liberal arts and integrating engineering and design as one job. Schoening, who joined last year, led product and design and shipped significant launches including Custom Agents for everyday knowledge workers and the Developer Platform allowing extensions. His background includes early design at Heroku, Cloud Run at Google, and six and a half years at GitHub working on Issues, Pull Requests, Actions, and cloud development environments used by millions. He advocates open customer-first platforms, and Notion’s open, agent-neutral context aims to benefit customers from small coffee businesses to large global enterprises balancing innovation, token costs, and governance at scale. As a two-time founder, Schoening embodies the multidisciplinary approach Notion values, and the company hires people whose work cuts across traditional discipline boundaries.
Deploy and manage coding agents at scale with the Unity Gateway CLI
Organizations face a challenge as new frontier models appear roughly every five days, making it difficult to balance flexibility with governance. Databricks introduces the Unity Gateway CLI, which lets admins centrally configure approved coding agents, models, MCP servers, skills, and spending policies. Developers launch any approved agent with a single ug command, and the CLI handles authentication and applies the published settings. Features include Smart Routing that selects cost-appropriate models per task, budget-aware defaults that switch to lower-cost options as spending thresholds are reached, and tracing to capture tool calls and skill invocations for waste analysis. In internal use, the system processed over 61 billion input tokens across roughly 360,000 requests, and tracing helped identify bugs that saved an estimated 1.2 million per year.
How Intelligent Load Shedding Prevents Cascading Failures in Tier-0 Systems
Salesforce’s Cloud Atlas team operates a globally distributed identity data store that must maintain five nines of availability, roughly five minutes of downtime per year. The team redesigned service protection after per-instance rate limiting proved insufficient under autoscaling and increasingly bursty, unpredictable agent-driven traffic. The new approach combines intelligent load shedding, which monitors queue time as an early indicator of system pressure before error rates spike, with coordinator-free global quota management. Servers independently measure and share customer usage to throttle noisy tenants without a centralized coordinator, preventing one customer’s surge from affecting others. This design prevents cascading failures by shedding lower-priority work before overload spreads across dependent services, ensuring graceful degradation rather than widespread outage.
From Dockerfile to Kit: the Docker Sandboxes Kit Specification
The Docker Sandbox Kit Specification v3 defines how an agent declares the networks, credentials, and volumes it needs, so these grants travel with the artifact rather than living in shell history or dashboards. A Kit is an ordinary OCI image whose manifest carries a descriptor annotation vnd.docker.sandbox.kit.descriptor, so it builds with docker buildx, pulls with docker pull, and gets scanned and signed by existing tooling. Capability types like network-policy@2 and credential@1 are versioned independently, and each normative statement in the specification is covered by a conformance check or a written waiver. Two Kit types exist: a workload supplies the root filesystem, while a mixin adds overlays such as a CLI with its network rule; the runtime rejects required capabilities it cannot satisfy. Composition uses strict provides/requires resolution, so authority changes appear as diffs, and a conforming runtime can gate upgrades that widen the grant set.
Efficient MoE Training for Biological Foundation Models
Mixture-of-experts (MoE) architectures scale model capacity more efficiently than dense transformers by activating only a subset of experts per token, but naive implementations suffer from fragmented expert computation, routing overhead, and memory pressure. NVIDIA Transformer Engine (TE) addresses these bottlenecks with optimized primitives including GroupedLinear for batched expert GEMMs, MXFP8 low-precision training, and a fused GroupedMLP kernel that combines quantization, SwiGLU activation, and routing-weight scaling. The BioNeMo recipe implements these techniques for biological foundation models, using TE’s autocast API for MXFP8 precision and the Sequential API to fuse the gate-up, ScaledSwiGLU, and down projections into a single operation. On eight NVIDIA B200 GPUs, the recipe achieved up to 2.21x the training throughput of the Hugging Face baseline for a Mixtral-8x7B model. The tutorial provides prerequisites, code examples, and configuration commands to reproduce the optimizations.
How Cloudflare addressed a cross-tenant data exposure vulnerability in Containers
Security researcher Oren Yomtov reported a vulnerability in Cloudflare Containers and Sandboxes through the bug bounty program, demonstrating that a Workers Paid account could recover residual disk blocks from previous containers on the same host. The issue arose from Linux device mapper thin provisioning with skip_block_zeroing enabled, which left unzeroed 64 KiB blocks when a 4 KiB write triggered allocation, allowing the remaining 60 KiB to retain prior data. Researchers recovered directory structures, database pages, and structurally complete SQLite databases from 18 of 24 placements across four continents, using ext4 checksums to distinguish foreign blocks. Cloudflare removed skip_block_zeroing, retired all running container disks, and cleared cached image snapshots created before the mitigation, completing cleanup by September 19. Retained disk-I/O telemetry showed no evidence of malicious exploitation, and the researchers confirmed secure deletion of recovered data.
Accelerating vision-language models with LFM2.5-VL-DSpark
Liquid AI released an experimental DSpark draft model for their vision-language model LFM2.5-VL-3B. The drafter adds a speculative decoding path that trades a minimal increase in memory footprint for a larger speedup without changing output quality. The 280M-parameter drafter, a 4-layer attention-only transformer, captures the target model’s hidden states at fixed tapped layers to draft a block of candidate tokens. On an M5 Max, decoding runs 2.30x to 3.13x faster by task, with end-to-end latency improvements of 1.56x to 2.62x. The drafter ships with day-one support for llama.cpp, MLX-VLM, and SGLang, and is available on Hugging Face in Safetensors and GGUF formats.
Self-encrypting VM images for confidential environments hands on
Confidential VM technologies such as AMD SEV-SNP and Intel TDX protect runtime data but leave persistent storage protection to the guest OS. This article demonstrates a method using Fedora Rawhide that combines dm-verity integrity for the immutable base image with a dm-crypt encrypted overlay for mutable parts. The base /usr partition remains verity-protected, while on first boot systemd-repart creates an encrypted root partition with a key sealed to the virtual TPM, ensuring each VM gets a unique key. An overlay filesystem mounts the immutable base under the writable overlay, avoiding a full copy of the base image and reducing boot time significantly in cloud environments. The article also outlines current limitations, including the lack of proof of origin for the overlay partition and the absence of integrity or replay attack protection.
Scaling GitOps: The "pull" architecture for global fleets
Red Hat conducted a performance and scale test of the argocd-agent pull architecture for GitOps, using 42 physical machines, 500 spoke clusters, and software including OpenShift 4.21 and OpenShift GitOps 1.20. The pull model decentralizes reconciliation to lightweight agents on each managed cluster, which pull configurations from a central hub, improving security by eliminating inbound administrative ports and reducing cross-cluster network traffic by up to 80%. In the bootstrap scenario, generating and syncing 31,200 applications across 480 clusters completed in 13 minutes 3 seconds, while the sync-to-commit scenario, which includes a polling delay, took 13 minutes. The test revealed bottlenecks including Kubernetes API limits on the hub, informer processing slowdowns causing watch restarts, event writer retry issues under load, and ApplicationSet controller RAM consumption. The team also deleted all 31,200 applications in 4 minutes 53 seconds using cascading deletion, and the insights are informing future OpenShift GitOps releases.
Continuing to Move PHP Open Source Forward
Each year, PhpStorm selects five open-source projects to sponsor for a full year, providing financial support and raising awareness of open source in the PHP community. For this year, the chosen projects are PHPUnit, Lerd, static-php-cli, Composer, and Larabelles, selected from community suggestions based on their impact and need for financing. Sebastian Bergmann, creator and maintainer of PHPUnit, offers the most widely used testing suite for PHP. George Dumitrescu’s Lerd manages local PHP installations and servers, while Jerry Ma’s static-php-cli is used by FrankenPHP and Native PHP. PhpStorm continues its sponsorship of the PHP Foundation and encourages other organizations to contribute to open source, as every bit of support makes a substantial difference when added together.
Designing the Kotlin Multiplatform and TeamCity Integration
The post describes the design of a new TeamCity integration for Kotlin Multiplatform (KMP) developers building iOS apps. The integration addresses common pain points: iOS builds require macOS infrastructure, Apple signing is complex, and there is no guided path from IDE to CI/CD. The solution provides a progressive onboarding flow within the IDE, starting with a tooltip after the first Git push. The flow generates four configuration files (teamcity.yaml, Fastfile, Appfile, Gemfile) and runs a build and tests on a hosted macOS agent, then guides the developer through a single form for Apple signing and TestFlight publishing, and finally automates the pipeline on GitHub pushes. TeamCity Cloud provides hosted macOS agents and a free starting tier, so no Mac is required. The design was validated through usability sessions and focuses on meeting the developer at the right moment with minimal configuration.
How I built agent-based security reviews on Databricks
Security reviews at Databricks relied on automation but still consumed expert time on predictable work, leaving less room for genuinely novel, high-risk architectures. The author built an agent-based layer on Databricks using Unity Catalog for governed tables containing standards, request data, and decisions; Databricks-hosted foundation models (Claude Haiku, Sonnet, Opus) for classification and reasoning; and Lakeflow Jobs for orchestrating notebook-based workflows on serverless compute. A conversational intake app built with Databricks Apps turns plain-language descriptions into structured requests and provides preliminary risk indications before a formal request is created. Seven focused agents each have a bounded responsibility—intake, risk assessment, requirements, specialized reviews, validation, workflow, and learning—ensuring behavior remains inspectable and testable. Routine requests that once waited days in the queue can now be completed in minutes, and security engineers spend more time on designs requiring genuine expertise.
Hitting a billion tokens per minute on one GPU by combining a query planner and an inference engine
Modal and CMU built Quail, a query-aware inference layer for AI-SQL that combines a query planner with an inference engine to process structured LLM transformations of database entries. On a multi-join query, Quail achieves over a billion tokens per minute per H100 GPU, over 10x faster than vLLM, costing under 6 cents per billion tokens on Modal. The gain comes from using the query plan to order requests for better KV cache management, including a revised Hydragen-style cascade attention and batching small requests to reduce host overhead. On a new AI-SQL benchmark, Quail runs 1.84x faster than vLLM, but falls behind on agent trace processing. The authors see this as part of a trend toward custom inference engines for specific workloads, with a reusable core like databases post-DataFusion, and note that much work was done by prompting coding agents.
How to Test and Fix Voice AI Agent Prompts with Twilio Conversation Relay and Cekura
This tutorial demonstrates building a voice AI agent with Twilio Conversation Relay, Node.js, and the OpenAI API, then testing and fixing its prompts using Cekura automated platform, a Twilio Startup Searchlight award winner. Conversation Relay manages real-time voice tasks like speech-to-text, text-to-speech, interruption detection, and turn management over WebSockets, while developers control business logic, tools, and AI integration. Cekura defines evaluation metrics, generates test conversations with realistic caller scenarios—interruptions, topic changes, and unexpected requests—and runs baseline tests to uncover prompt gaps. After analyzing failure reports, developers update the system prompt and re-run tests; fixing a fee-waiver bug raised the pass rate from below 70% to the 90-95% target in one refinement. The tutorial emphasizes that prompts are source code requiring version control and automated testing for reliable voice AI agent behavior.
Measuring the back/forward cache with Application Metrics
The back/forward cache (bfcache) allows instant page restores by freezing snapshots, yet its hits register no events in traditional telemetry, creating a blind spot for teams. Sentry’s bfcacheMetricsIntegration fills this gap by modeling three browser-provided signals as application metrics: a navigation counter with hit/miss outcome, a per-reason counter for misses via the notRestoredReasons API, and a distribution of fallback reload duration. The integration is opt-in and launched in JavaScript SDKs v11; detailed miss reasons are currently Chromium-only due to browser API support. Users can query these metrics to break down bfcache health by route, release, or browser, enabling discovery of regressions caused by unload listeners, cache-control headers, or third-party scripts. This turns the bfcache from an invisible performance feature into a measurable, trackable signal that teams can monitor and improve.
A Practical Recipe for Semi-Supervised Federated ASR: Online Pseudo-Labels with Server Update Stabilization
Semi-supervised federated learning (SSFL) for ASR uses a teacher to generate pseudo-labels on unlabeled client data while the server trains on a small labeled seed. The paper finds that pseudo-label errors compound across the output sequence and training rounds, causing divergence and a large gap to fully-supervised FL. Two coupled design axes close this gap: the teacher type (per-client online, global broadcast, or transitioning) and the anchor (server-side labeled-data updates that stabilize training). A per-client online teacher diverges on its own but, once stabilized, matches or beats the global teacher in-domain and under domain shift. The server must keep training on labeled data between rounds; otherwise the online teacher drifts, and this interleaving governs convergence more than the seed model does. The proposed guidelines improve over the strongest prior method on 9 of 11 pairs, by 20.8% in-domain and 10.0% cross-domain on average.
Turn your REST APIs into MCP tools with Google Cloud API Gateway
Most enterprise REST APIs are invisible to AI agents, forcing teams to run separate MCP servers that duplicate their existing gateway logic. Google Cloud API Gateway in Public Preview solves this by acting as a remote MCP server — annotate the OpenAPI 3.0.x/3.1.x spec with x-google-api-management.mcp and deploy it with no extra infrastructure. The gateway accepts MCP JSON-RPC requests on a single /mcp endpoint, transcodes tools/call into the corresponding REST call, runs it through existing policies, and returns the response as an MCP result. Authentication via JWT or API key works unchanged for REST operations; tools/list is unauthenticated by default and requires JWT in production for security. With no separate server to operate, teams in Public Preview can expose up to 1,000 tools per gateway that support existing authentication and quota, and the MCP server is automatically discoverable via API hub and Agent Registry.
Reproducing OLMo 3 7B Pre-training in MaxText: case study of large scale training on TPUs
The post describes the reproduction of AI2’s OLMo 3 7B pre-training using MaxText on Google Cloud TPUs, covering both stages, matching the reference loss curve and held-out metrics. PyTorch weights were converted to JAX with logit parity, KL divergence 1.5e-3 and 98.75% top-1 agreement, and held-out evaluations caught a data loader bug that made incorrect gains appear. The run demonstrated reliability via exact checkpoint-resume with zero loss delta, resilience to a 75% capacity loss via resize with 1% throughput preservation, and seamless TPU generation change. Performance optimizations achieved 44.5% MFU on Ironwood, and head reshaping from 32 heads by 128 to 16 by 256 yielded 12.4% faster throughput at identical parameters and FLOPs. The post emphasizes that faithful reproduction requires verification on generalization metrics and provides instructions for readers to reproduce the run with open sourced code.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.