Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
The Convergence of Open Table Formats and Open Catalogs: Catalog Commits is Generally Available
Catalog Commits is generally available for Unity Catalog managed Delta tables, aligning Delta with Iceberg’s catalog-oriented model and making the catalog responsible for table discovery, access, and latest table state. The change addresses three coordination problems: metadata “split brain” when engines write directly to storage, fragmented multi-engine governance, and the historical inability to coordinate atomic writes across multiple tables. With Catalog Commits enabled, Unity Catalog brokers table access through standardized APIs, keeping catalog and table state synchronized and enabling consistent authorization, holistic auditability, automated optimizations, and multi-statement, multi-table ACID transactions on Databricks. The release supports Databricks products and engines including Delta Spark, Delta Flink, Starburst Trino, DuckDB, and StreamNative, while Delta Kernel provides a shared path for connector support.
Announcing Databricks student fellows
Databricks is launching the Student Fellows Program, an opportunity for university and college students interested in computer science, AI, and data engineering to become leaders in their campus communities. Fellows receive specialized training from Databricks experts, build hands-on skills with the Databricks platform, and organize campus activities such as hackathons, tech talks, and study groups while connecting peers with resources. The initiative also offers opportunities for free or discounted certification exam vouchers and practical experience intended to support applications for internships with Databricks, its customers, or partners. Student Fellows’ primary mission is to foster a community of learners and serve as a bridge between Databricks and their university communities, with the first cohort currently accepting applications.
Faster Queries and New Capabilities with the Open-Source Databricks JDBC Driver
The open-source Databricks JDBC driver 3.x and later releases address connectivity needs affecting dashboards, spreadsheets, applications, and other data workflows. Compared with the legacy 2.x driver, they deliver up to 30% faster retrieval of large query results while preserving backward compatibility. Architectural changes add full Arrow-based transfer for JDK 16+, a first-class asynchronous execution API, stream-based ingestion into Databricks Volumes, and integration with the Statement Execution API. The driver also supports stored procedures, multi-statement transactions, Unity Catalog metric views, query tags, geospatial and complex data types, plus built-in client telemetry for query latency, metrics, and errors. Its open-source ownership and Databricks maintenance are presented as enabling faster fixes, new features, transparency, contributions, and tighter platform integration.
How we achieved truly serverless GPUs
GPU inference workloads suffer from spiky demand, leading to low GPU Allocation Utilization when capacity is fixed. Modal reduced replica startup latency from over 2 kiloseconds to about 50 seconds using four techniques. A buffer of idle GPUs removes instance allocation from the hot path. A custom content-addressed filesystem serves container images lazily from a multi-tier cache. CPU and CUDA checkpoint/restore fast-forward through initialization by directly restoring processes and GPU contexts into memory. These optimizations enable truly serverless GPU inference, as demonstrated by Reducto’s document processing scaling to thousands of GPUs with cold starts reduced from ~70s to ~12s.
How to Build Low-Code API Integrations for Enterprise Apps Using Okta
Okta’s API Integration Actions let independent software vendors build Provisioning, Entitlements, and Universal Logout integrations in the Okta Integration Network using Workflows, Okta’s low-code builder. The approach maps existing APIs directly to Okta action contracts, allowing developers to avoid building and maintaining a SCIM server while translating API calls into formats Okta can consume. The guide walks through creating an OIN integration, configuring tenant and authentication mappings, adding API Integration Action components, building and validating flows, and submitting them through the OIN Wizard. It also explains how to create and update an org-only test instance before review, noting that submission changes do not automatically update an existing test instance for security reasons.
We replaced Redis with MySQL for inventory reservations—and it scaled
Shopify replaced its Redis-based inventory reservation system with MySQL to unify reservations with the inventory ledger and provide ACID coordination between payment and inventory updates. The new design stores one row per sellable unit in a bounded pool of up to 1,000 rows per item/location combination, uses MySQL 8's SKIP LOCKED to avoid waiting on locked units, and replenishes depleted pools. Composite primary keys reduced row locks, READ COMMITTED avoided gap locks that interfered with replenishment, and consistent lock ordering prevented deadlocks. The investigation found connection-pool usage elsewhere in checkout, rather than reservation query speed alone, was the immediate ceiling; cleanup removed 50% of primary-database reads and 33% of transactions, while configuration changes added headroom. Shopify validated the design in shadow mode, then rolled it out gradually with a Redis rollback path, meeting high-throughput targets during peak 2025 traffic.
Building Blocks for Foundation Model Training and Inference on AWS
The post explains how foundation-model scaling now spans pre-training, post-training, and test-time compute, creating shared infrastructure requirements rather than a single pre-training curve. It examines how AWS infrastructure and open-source tools support these workloads across accelerated compute, networking, storage, orchestration, ML frameworks, and observability. Technical detail includes NVIDIA H100, H200, B200, and B300 GPU characteristics, NVLink/NVSwitch scale-up communication, and EFA-based scale-out networking for distributed collectives. The architecture also incorporates EC2 P instances, tiered storage, Slurm, Kubernetes, SageMaker HyperPod, CUDA, NCCL, PyTorch, Prometheus, and Grafana. The conclusion presents integration points between these layers as the basis for diagnosing bottlenecks and making scaling decisions across the foundation-model lifecycle.
Unlocking the Archives: Turning Unstructured Documents into a Searchable Database for Groundwater Discovery
MapAid and the Sudan Association for Archiving Knowledge (SUDAAK) needed to make nearly 700 scanned geological reports searchable and recover well data for MapAid’s WellMapr groundwater models. Databricks built a Databricks-only pipeline that renders pages as images, uses multimodal AI Functions for page classification and schema-constrained JSON, and applies targeted sampling to cut classification processing by more than 70%. Water-relevant documents receive page-by-page OCR through the Foundation Model API, with well and borehole identifiers linking information across pages before structured records capture locations, depths, static water levels, and yields. In its first full run, 654 documents and 5,570 pages were classified in under three hours; 95% received excellent or good evaluations, about half contained water data, and 299 well or borehole records were extracted.
Fighting Tool Sprawl: The Case for AI Tool Registries
Rapid enterprise AI agent adoption without centralized infrastructure introduces widespread tool sprawl, duplicated engineering effort, and severe operational opacity. Individual application teams frequently build ad hoc, undocumented integrations, leaving security teams unable to discover, audit, or approve active agent capabilities. Internal enterprise tool registries address this coordination failure by establishing an organization-specific catalog tailored to internal security policies, data, and compliance requirements. A production-grade registry delivers core discovery and version tracking functionality while surfacing security certification metadata and enabling consistent identity-scoped access control across teams. Organizations establishing centralized tool infrastructure can accelerate agent onboarding, implement deny-by-default governance policies, and mitigate technical debt before uncoordinated deployments compound across the enterprise.
Beyond efficiency: PayPal expands what's possible to build with AI
PayPal describes how it used Cursor to accelerate software development across an organization of 8,000 developers and a codebase spanning decades. It began with high-impact teams because the company moves money for more than 400 million customers and mistakes are costly; within two weeks, those teams were deploying daily rather than weekly or biweekly, and adoption spread organically. The company says a Java upgrade across 3,000 applications took two months instead of an estimated eight to twelve, while highly adopting teams saw more frequent deployments, shorter lead times, and lower change failure rates. PayPal continues to measure deployment frequency, lead time, and change failure rate rather than AI-generated code, and Michelle Chance believes the company could deliver 40% more capabilities in 2026 than in 2025.
Updates to Bugbot for Teams and Individuals
Bugbot is replacing its $40-per-seat monthly subscription with usage-based billing for Teams and Individual plans. Existing customers move to the new model at the first billing renewal after June 8, 2026; an annual subscription purchased in May 2026, for example, changes in May 2027, while earlier switching is available in the Cursor dashboard. Teams are billed from on-demand spend, Individuals from included usage, and an average run costs $1.00–$1.50 depending on pull request size and complexity. Usage billing also adds configurable effort levels, including deeper reviews and custom logic for selecting effort: the default retains an 80% bug-resolution rate by merge time, while high effort finds 35% more bugs in internal runs with that rate unchanged.
Predictive quality starts where defect detection stops
Manufacturing quality teams often receive defect-rate reports after the conditions behind them have changed, because inspection, supplier, and environmental data are disconnected. Predictive quality combines production, inspection, and supplier data with machine learning to forecast defects before final inspection, shifting quality management from reactive documentation to proactive intervention. Databricks Genie is presented as a natural-language interface for querying those sources together, including questions about first-pass yield, supplier lots, root-cause contributors, and process conditions; its answers include citations and can surface unusual patterns. The described capabilities include contextual understanding of terms such as NCR, CAPA, and CPK threshold, multi-source reasoning, and traceable outputs tied to records. The intended outcome is faster analysis and earlier action to reduce scrap before its cost is incurred.
Retail markdown optimization: from reactive markdowns to proactive
Retail markdown optimization addresses the gap between changing demand signals and delayed merchandising action. The post defines it as strategically reducing prices on slow-moving or end-of-life inventory, using demand forecasts, sell-through rates, weeks of supply, and price-elasticity models to choose which SKUs to mark down, when to start, how deep to discount, and where to act. It presents Databricks Genie as a natural-language interface across e-commerce, store, and wholesale data, with supplier, margin, and historical-pattern context; one example is Coop’s Microsoft Teams assistant, AskCap, which reported a 30% retention rate among internal users. Earlier detection of sell-through deceleration can give merchants more time to protect margin, adjust open-to-buy, and reallocate capital, while Genie supports decisions rather than making them.
From vibe code to production-ready: observability for Next.js and Supabase apps
AI-assisted software development often generates functional code that lacks proper instrumentation, leading to unindexed database queries, N+1 patterns, and fragmented connection strategies. While Supabase provides built-in query analytics, security advisors, and subsystem logs, it cannot trace distributed requests across the entire application stack. Integrating Sentry bridges this visibility gap by establishing distributed traces across Next.js frontends, Deno-based Supabase Edge Functions, and Postgres databases. Teams can forward Supabase logs into dedicated Sentry projects and configure runtime integrations to automatically detect performance bottlenecks such as slow spans and Web Vitals regressions. Furthermore, integrating Model Context Protocol servers allows Sentry's Seer debugger to identify root causes and generate automated fixes for emerging issues.
ML based ranking using Nrtsearch
Nrtsearch’s Inference Plugin embeds machine-learning ranking in the Lucene-based search engine, addressing scalability problems in a two-stage design where a separate inference service scored candidates. Previously, large candidate sets and feature payloads created network-transfer and serialization overhead; the new flow extracts features from index documents or request parameters and scores documents within replica nodes. An ML Scorer selects models, transforms features, runs MLeap-based inference, and aggregates scores, with support for XGBoost, neural networks, multi-model execution, batch inference, and custom Java scorers. Models are loaded from MLflow through cluster configuration, while teams can use Lucene expressions or implement modular interfaces; testing, canary deployment, rolling updates, and Prometheus metrics support operations, and future work includes GPU-based ranking and additional inference engines.
Powering self-driving vehicle analytics at Avride with ClickHouse Cloud
Avride’s expanding autonomous passenger vehicle and delivery robot fleet generated thousands of data points per minute, overwhelming its legacy Apache Iceberg and Parquet metadata indexing architecture. Iceberg’s optimistic concurrency model struggled with parallel writes, pushing ingestion latencies to hours or days, inflating query times to 10–20 seconds, and requiring raw data duplication across storage locations. To resolve these bottlenecks, Avride built a custom indexing and metrics warehouse powered by ClickHouse Cloud on AWS backed by Amazon S3 object storage. The new architecture dropped ingestion delays to seconds, reduced worst-case lookup latencies to under one second, eliminated the need for bespoke caching layers, and removed payload duplication. Beyond operational analytics, the unified ClickHouse platform now supports internal ride-replay debugging tools, simulation workflows, and high-cardinality vehicle C++ performance trace profiling.
Upgrading M3DB from v1.1 to v1.5
Uber upgraded M3DB from v1.1 to v1.5 after three years on its previous open-source deployment, seeking a repeatable rollout process for a roughly 100-cluster, 5,500-node environment. The effort addressed scale-specific validation: offline benchmarks measured correctness, latency, success rates, bootstrap and tick durations, and persistence across restarts and replacements, while online shadow testing compared sampled production traffic between old and new versions. Within each cluster, upgrades were limited to one node at a time because writes used a two-thirds majority quorum. A rollout-related read-quorum incident exposed missing replica data; the cause was bootstrap misconfiguration, corrected by using “exclude_commitlog,” since production workloads did not employ WAL. Production then showed significantly lower memory usage and improved node bootstrap times, supporting continued, data-driven upgrades and contributions to M3DB's benchmarking and documentation.
Using MemAlign to Improve Evaluation of Traditional Machine Learning in Genie Code
Genie Code’s traditional machine learning workflows are difficult to evaluate because notebook quality depends on code, best practices, and adaptation to data and task. The team built an evaluation pipeline with nine LLM judges, each scoring applicable dimensions—including data exploration, imputation, training, MLflow logging, and cell organization—from 1 to 3, using test cases across classification, regression, and forecasting. On 50 human-graded notebooks, initial judge alignment was weak in several areas, including model training (MAE 0.680) and model use (0.562), with positivity bias contributing to missed technical flaws. Applying MLflow’s MemAlign, which combines generalized semantic guidelines with retrieved episodic examples, reduced judge error by 74–89% on the most important dimensions; removing episodic memory weakened most gains, while data exploration showed a nonsignificant regression.
How Superhuman and Databricks built a 200K QPS inference platform together
Superhuman replaced a DIY vLLM-based serving stack for its grammar-correction model with Databricks model serving, targeting peak traffic above 200,000 QPS, sub-second P99 latency, four-nines reliability, and no quality regression. The platform combines an Endpoint Discovery Service with power-of-two-choices load balancing, concurrency-based autoscaling, and asymmetric scale policies to reduce hotspots and handle rapid traffic ramps. Lazy-loading container images changed startup from several minutes to a few seconds by fetching metadata first and retrieving 4MB blocks on demand. Runtime work on H100 GPUs raised per-pod throughput from 750 to 1,200 QPS: FP8 weight quantization contributed up to 30%, while a multiprocessing server added 20%, alongside C++ and asynchronous scheduling optimizations. The teams validated changes through shadow testing and evaluation harnesses, and Superhuman is migrating additional models and workflows to the platform.
Enhancing Ad Relevance: Integrating Real-Time Context into Sequential Recommender Models
Pinterest's sequential ads candidate generator inferred user embeddings offline from historical offsite conversions, which lacked real-time browsing context. Consequently, less than 1% of impressions on the Related Pins surface were attributed to this generator because its retrieved candidates struggled in downstream ranking. To resolve this limitation, engineers developed the Contextual Sequential Two Tower Model, integrating a context layer directly into the query tower alongside user demographic features. Offline training utilized synthetic context derived from positive conversion labels paired with high dropout rates to preserve sequence reliance, while serving split into offline Transformer inference and online context computation. Offline evaluations showed a 3x to 10x increase in Recall@K, and production deployment lifted median candidate relevance by 275% to 300% and overall ROAS by approximately 0.7%.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.