---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [How Databricks Genie improves retail personalization](https://yomu.fyi/post/how-databricks-genie-improves-retail-personalization.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Sarah Duffy
- Published: May 20, 2026
- Type: Explainer

Retail personalization often stalls when CX leaders must wait for analysts to answer questions about segment behavior, loyalty performance, channel preference, or churn risk, narrowing the window for action. Databricks Genie addresses this access gap by letting business users query unified structured and unstructured enterprise data in plain English instead of SQL. Its retail differentiators include identity-resolved queries across channels and devices, lifecycle-stage awareness, campaign response and control-group data integration, and privacy controls enforced through Unity Catalog. The source says Genie reduces routine analyst requests rather than replacing data science teams, while enabling merchandisers, category managers, loyalty marketers, and CX leaders to self-serve operational questions. It cites 7-Eleven’s use of Databricks SQL, Unity Catalog, and AI/BI Genie to launch, refine, and measure personalized offers within a secure, unified platform.


### [Databricks for Good and Virtue Foundation: Partnering to Connect Medical Volunteers to Critical Health Services in 72 Countries](https://yomu.fyi/post/databricks-for-good-and-virtue-foundation-partnering-to-connect-medica.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Priyanka Mehta, Shaunak Sen
- Published: May 20, 2026
- Type: Problem & solution

Virtue Foundation’s VF Match platform connects medical professionals with volunteer opportunities across 72 low and low-middle income countries, but scaling its initial LLM proof of concept into production required a robust data pipeline. The Databricks-based Foundational Data Refresh combines Overture Maps and Bright Data, processes more than 25 million web pages with targeted GPT extraction steps, and uses Spark, Photon, and Lakeflow Jobs to orchestrate checkpointed, modular workloads. Splink resolves duplicate facilities and nonprofits across messy sources; Photon reduced the slowest entity-resolution partition from 30 minutes to approximately two minutes, a 15x improvement. A prototype VF Agent adds natural-language access through LangGraph, Databricks Model Serving, AI Search, and Genie, while the resulting system provides an up-to-date global view of healthcare infrastructure and volunteer needs.


### [Introducing Arrow UDFs in PySpark: A Faster, Leaner Replacement for Pandas UDFs](https://yomu.fyi/post/introducing-arrow-udfs-in-pyspark-a-faster-leaner-replacement-for-pand.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Ruifeng Zheng, Yicong Huang
- Published: May 20, 2026
- Type: Explainer

Apache Spark’s Pandas UDFs improve Python UDF throughput through Arrow-based serialization and batching, but Pandas/Arrow conversion can add copies, limit zero-copy behavior, and weaken support for complex datatypes. Native Arrow UDFs, introduced in Databricks Runtime 18.0, operate directly on Arrow data without converting inputs to Pandas or NumPy objects, preserving columnar layout and enabling vectorized processing through Arrow’s compute and memory model. The new @arrow\_udf decorator supports scalar, aggregate, and table functions, while type-hinted @udf definitions, mapInArrow, and applyInArrow extend the interface to DataFrame and grouped operations; iterator-based modes can amortize initialization costs. In a comparison, an Arrow UDF ran about 10% faster than a Pandas UDF and used about 40% less memory, with better complex-datatype support.


### [Scaling reinforcement learning at Applied Compute](https://yomu.fyi/post/scaling-reinforcement-learning-at-applied-compute.md)
- Company: [Modal](https://yomu.fyi/company/modal.md)
- Published: May 20, 2026
- Type: Problem & solution

Applied Compute, founded by former OpenAI Codex and o1 researchers, builds custom AI agents for enterprises through a post-training approach they call Specific Intelligence. Their core training mechanism is Reinforcement Learning, where agents repeatedly attempt tasks in replayable environments and are scored against reward functions to refine model weights. For clients like DoorDash and Cognition, this means training specialized agents for merchant onboarding and bug-catching that improve from continuous use. The company evaluated multiple infrastructure platforms before selecting Modal, which provides ephemeral sandboxes for complex environment simulation, sub-second cold starts to keep GPUs busy, and serverless fan-out for parallel grading. Modal’s primitives allow each phase of the RL loop—rollouts, evals, and inference—to operate with appropriate resources while sharing state efficiently.


### [Improvements to Cursor Automations](https://yomu.fyi/post/improvements-to-cursor-automations.md)
- Company: [Cursor](https://yomu.fyi/company/cursor.md)
- Published: May 20, 2026
- Type: Announcement

Cursor Automations are now available in the Agents Window as well as the existing Automations interface, allowing users to create and manage them alongside agents in the same workspace. The release also lets an automation attach multiple repositories, so agents can use context across codebases to deliver, test, and verify tasks, or run with no repository when the work is not code-centered. For no-repo use cases, five Cursor Marketplace templates cover Slack digests, product analytics, FAQ drafting from Slack and documentation context, finance reporting from providers such as Stripe, and customer health monitoring across tools including Granola, Slack, and Databricks. Newly created automations receive a 50% discount on all agent runs for the next seven days, while the release directs users to the documentation for more information.


### [Six Steps to Improve Your Sender Reputation](https://yomu.fyi/post/six-steps-to-improve-your-sender-reputation.md)
- Company: [Resend](https://yomu.fyi/company/resend.md)
- Published: May 20, 2026
- Type: Tutorial

Sender reputation is the largely private assessment that inbox service providers such as Gmail, Outlook, and Yahoo make before deciding where to deliver email. It reflects infrastructure compliance, sending history and consistency, list hygiene, mail frequency, and recipient engagement, with providers increasingly emphasizing domain reputation over IP reputation. The guide recommends six recovery steps: audit SPF, DKIM, and DMARC; warm up new domains gradually; identify provider-specific blocks or blocklists; clean and segment lists; make unsubscribing easy; and send valuable, mobile-friendly content. It also advises using double opt-in, monitoring engagement, bounce and spam responses, throttling, and DMARC reports, while tightening policies or requesting provider reviews after fixes. Because providers continually reevaluate senders, reputation management requires repeated observation, testing, and adjustment rather than a one-time repair.


### [Welcoming Tatira Andrade, our new Executive Assistant](https://yomu.fyi/post/welcoming-tatira-andrade-our-new-executive-assistant.md)
- Company: [Resend](https://yomu.fyi/company/resend.md)
- Published: May 20, 2026
- Type: Announcement

Tatira Andrade is joining Resend as an Executive Assistant, bringing nearly 16 years of experience in executive support, complex logistics, and process optimization. Her background includes working with companies such as Nubank, where she helped leadership teams streamline daily operations and scale workflows efficiently, while her interests in travel, geography, languages, and organization align with the role. Andrade says she is motivated by making processes and experiences more seamless; she favors Gemini, uses Command + Shift + 4 for screenshots, and currently names Montreal in autumn as a favorite destination. Her advice to ambitious executive assistants is to keep learning, build technology and language skills, stay updated, and remain genuinely curious about their companies and colleagues.


### [Optimizing Our Build Times by Migrating from Webpack to Rspack](https://yomu.fyi/post/optimizing-our-build-times-by-migrating-from-webpack-to-rspack.md)
- Company: [Yelp](https://yomu.fyi/company/yelp.md)
- Author: Benson Pan, Software Engineer
- Published: May 20, 2026
- Type: Problem & solution

As Yelp's JavaScript monorepo expanded over time, Webpack build speeds became a significant engineering bottleneck. To resolve this without rewriting complex configurations, the team selected Rspack for its direct compatibility with existing Webpack setups and plugins. Yelp implemented an adapter pattern to generate Rspack configurations dynamically from original Webpack files, enabling a staged opt-in rollout across development tools and CI environments. Following the initial migration, engineers addressed performance issues in barrel files using automated codemods to replace star re-exports and import-then-export patterns with named re-exports. Combined with Rspack's portable persistent caching, the migration reduced integration build times by approximately 52 percent and warm cache builds by up to 80 percent.


### [Automate Data & KPI Monitoring with SQL Alerts](https://yomu.fyi/post/automate-data-kpi-monitoring-with-sql-alerts.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Srilekha Dornadula
- Published: May 19, 2026
- Type: Announcement

Databricks SQL Alerts is generally available as an automated alternative to manually checking dashboards and rerunning monitoring queries for business KPIs, pipeline health, and data quality. Teams define a SQL query, evaluation condition, schedule, and notification destinations, then receive alerts when results cross configured guardrails; standalone alerts can also run within Lakeflow Jobs. GA supports authoring in the SQL editor, notifications through email, Slack, PagerDuty, Microsoft Teams, or webhooks, and production management through Git, Declarative Automation Bundles, APIs, Terraform, and SDKs. The post’s revenue example compares yesterday’s result with a seven-day average and triggers when revenue\_pct\_change is below -5%, while the SQL Alert task can expose OK, TRIGGERED, or ERROR to downstream pipeline logic.


### [How to Build Real-Time Fraud Detection using Spark Real-Time Mode and Lakebase](https://yomu.fyi/post/how-to-build-real-time-fraud-detection-using-spark-real-time-mode-and.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Sixuan He, Navneeth Nair
- Published: May 19, 2026
- Type: Tutorial

Card fraud operates in seconds, while organizations need to score and block suspicious transactions before settlement without adding a separate streaming stack. This Solution Accelerator presents an open-source, end-to-end implementation on Databricks, covering transaction ingestion, Spark Real-Time Mode processing, fraud scoring, Lakebase feature serving, and a Databricks Apps monitoring dashboard. RTM processes streaming workloads in milliseconds and targets sub-300ms latency, while Lakebase supplies a managed, serverless Postgres serving layer with sub-millisecond reads. The design keeps batch ETL, training, streaming, and governance within the same platform. Deployment progresses from a synthetic rate-source quick start to Kafka-based parsing, stateful per-card velocity tracking with TTL, enrichment, weighted scoring, routing, and optional MLflow-trained RandomForest scoring, followed by a Streamlit app for live decisions.


### [How Databricks Genie improves supply chain visibility with real-time AI analytics](https://yomu.fyi/post/how-databricks-genie-improves-supply-chain-visibility-with-real-time-a.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Caitlin Gordon
- Published: May 19, 2026
- Type: Problem & solution

Supply chain leaders often have extensive operational data yet still respond reactively because predictive signals such as supplier lead-time trends, inventory velocity, weather, and commodity prices remain siloed and difficult to synthesize. Databricks Genie is presented as a plain-language intelligence layer that lets leaders interrogate operational and external data in the flow of work rather than relying on analyst-led BI sessions. Users can ask questions combining supplier tiers, lead-time changes, inventory coverage, contract terms, and production schedules, then follow with what-if queries about financial exposure if conditions worsen. The post says Genie returns answers in seconds, supports near-real-time monitoring and proactive alerting, and enables shared, governed answers for procurement, operations, and finance, shifting decisions from reactive reporting toward earlier evidence-based action.


### [OlmoEarth v1.1: A more efficient family of Earth observation models](https://yomu.fyi/post/olmoearth-v1-1-a-more-efficient-family-of-earth-observation-models.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Kyle Wiggers
- Published: May 19, 2026
- Type: Announcement

OlmoEarth v1.1 is a new family of Earth observation models designed to reduce the compute required for large-scale satellite imagery predictions. Compute dominates the lifecycle from data export through preprocessing, inference, and post-processing, motivating a focus on transformer token sequence length. For Sentinel-2, the models collapse three resolution-specific tokens into one, reducing token counts threefold; this initially caused a 10 ppt drop on the m-eurosat kNN benchmark, so the pre-training regimen was modified. OlmoEarth v1.1 runs up to three times cheaper than v1 while maintaining similar performance across a mix of research benchmarks and partner-built tasks, although some regressions remain, and the same dataset is used to isolate methodological effects.


### [A CFO’s guide to managing value-based care financial performance](https://yomu.fyi/post/a-cfo-s-guide-to-managing-value-based-care-financial-performance.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Adam Crown
- Published: May 19, 2026
- Type: Explainer

Value-based care changes healthcare finance by tying payment to outcomes, cost efficiency, and equity rather than service volume, while shifting greater financial risk to providers under ACO, bundled, and capitated contracts. A CFO must therefore track attributed populations, PMPM cost trends, utilization, clinical outliers, quality thresholds, and contract performance against benchmarks. Databricks Genie is presented as a conversational interface over integrated attribution, claims, clinical, and benchmark data, allowing questions such as PMPM divergence among high utilizers and DRG drivers to be answered in real time. The source also identifies incorrect attribution and risk-adjustment coding errors as major threats, and describes predictive risk stratification and quality-gap forecasting as ways AI can support earlier intervention.


### [Scaling Airbnb’s identity graph with a unified knowledge graph infrastructure](https://yomu.fyi/post/scaling-airbnb-s-identity-graph-with-a-unified-knowledge-graph-infrast.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Lucen Zhao
- Published: May 19, 2026

Airbnb migrated its massive identity graph from a third-party graph database vendor to a unified, in-house knowledge graph platform to solve long-tail latency and scaling bottlenecks. Built on JanusGraph with AWS DynamoDB for persistence and OpenSearch for indexing, the new architecture decouples storage operations from graph traversal logic. The migration improved P99 query latency, eliminated routine instance reboots, and supported a tenfold increase in write throughput.


### [Stop rogue AI: How Unity Catalog secures your agent actions](https://yomu.fyi/post/stop-rogue-ai-how-unity-catalog-secures-your-agent-actions.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Ahmed Bilal
- Published: May 19, 2026
- Type: Problem & solution

Agentic AI systems connected to external tools can take destructive production actions while operating within delegated authority, yet without restrictions on tool selection or records of their calls. Unity Catalog now governs MCP tools with fine-grained access control, service policies, and payload logging, while Unity AI Gateway enforces decisions in real time. Policies are SQL functions receiving actor and context information; they can allow, deny, or require user consent, inspect arguments and caller properties, and block calls before execution. Every tool call is recorded in a Unity Catalog-managed Delta table with the tool name, arguments, result, user identity, and allow-or-deny status, making the activity queryable with SQL. The service policies and payload logging are available as a Gated Beta, with early access offered through Databricks account teams.


### [Ready for your busiest day: How we scale](https://yomu.fyi/post/ready-for-your-busiest-day-how-we-scale.md)
- Company: [Intercom](https://yomu.fyi/company/intercom.md)
- Author: Ryan Sherlock
- Published: May 19, 2026
- Type: Explainer

Fin describes how its customer service platform is designed to absorb high-volume events while preserving customer isolation, availability, and operational control. At daily peaks, it handles over 150,000 customer requests per second and more than 70,000 asynchronous requests per second; its source-of-truth data spans 128 Vitess shards, while Elasticsearch supports over 650TB, 1.7 trillion documents, and peaks above 40,000 requests per second. The architecture favors AWS primitives, Vitess managed by PlanetScale, online and incremental migrations, fair queues, feature flags, heartbeat metrics, and automatic rollback, with roughly 250 production ships per workday and merge-to-production time under 10 minutes. The stated conclusion is that scale depends less on headline capacity than on clear scaling levers, tenant fairness, customer-outcome monitoring, reversible change, and routine infrastructure improvement without planned customer-impacting downtime.


### [Databricks context engineer associate: the industry’s first certification for reliable AI agent systems](https://yomu.fyi/post/databricks-context-engineer-associate-the-industry-s-first-certificati.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Rachel Canetta, James Kantor, Trang Le
- Published: May 19, 2026
- Type: Announcement

Databricks announces the Databricks Certified Context Engineer Associate, described as the industry’s first certification purpose-built for context-aware AI systems. It assesses whether practitioners can design, assemble, and govern the information AI agents receive at inference time, including system prompts, retrieval, memory, external tools, context-window management, and governance. Candidates work with Databricks technologies such as AI Search, Lakebase, MLflow, MCP, and Unity Catalog, while advanced coverage includes multi-agent systems, long-horizon workflows, and evaluation of context decisions. The beta exam will be available free to Data + AI Summit attendees, who may take it once; beta results are expected in 6–8 weeks.


### [Scaling Real-Time Traffic Forecasting with a Graph-Aware Transformer](https://yomu.fyi/post/scaling-real-time-traffic-forecasting-with-a-graph-aware-transformer.md)
- Company: [Uber](https://yomu.fyi/company/uber.md)
- Published: May 19, 2026
- Type: Explainer

DeepETT is Uber’s real-time traffic forecasting system for estimating traversal times on roughly 100 million road segments, supporting routing and arrival-time estimates. It replaces a legacy blend of recent observations and historical patterns with a deep-learning model designed for rapidly changing conditions, sparse roads, and long trips. To keep deployment predictable at global scale, the system uses fixed-size computation over pre-aggregated segment, graph, region, hourly, weekly, and event views rather than a dynamic graph, while predicting segment ETTs independently of routing. Training targets segment-level mean squared error, while trip-level mean absolute error is evaluated offline after routing sums segment forecasts; calibration and continual real-time correction address downstream error. In a global A/B test, long-trip arrival-time accuracy improved 6%, variance explained rose 19%, navigation defects fell 2.73%, and mobility gross bookings increased by an estimated $100 million annually, with forecasts exceeding 2 million per second.


### [What's new in ClickStack - April 2026](https://yomu.fyi/post/what-s-new-in-clickstack-april-2026.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: The ClickStack Team
- Published: May 19, 2026
- Type: Announcement

ClickStack's April release introduces several core enhancements across querying, alerting, and dashboard visualization capabilities. The platform added SQL-powered alerting, allowing teams to express complex statistical rules like rolling baselines and anomaly detection using native ClickHouse SQL and window functions without external pipelines. To improve search and filtering performance across common observability workloads, the default otel\_logs schema was redesigned with ClickHouse text indexes and adjusted ordering keys. Furthermore, heatmaps transitioned into a first-class chart type inside the shared visualization system for latency distribution tracking. The update also delivers persistent alert execution error reporting, expanded threshold comparison operators, and per-series number formatting across dashboards.


### [How to safeguard AI workloads with Unity AI Gateway Guardrails](https://yomu.fyi/post/how-to-safeguard-ai-workloads-with-unity-ai-gateway-guardrails.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Tim Lortz
- Published: May 19, 2026
- Type: Tutorial

Unity AI Gateway’s beta LLM Guardrails extend earlier Gateway guardrails with LLM-powered evaluation, pre-built protections, and tunable custom rules for AI security, compliance, and sensitive-data protection. An Acme marketing scenario maps policies to PII redaction on inputs, jailbreak and prompt-injection blocking, unsafe-content blocking on outputs, and a custom rule that blocks competitor references. Teams configure these guardrails on an endpoint, optionally use Log mode, select evaluator endpoints, and test representative requests while tracking activity in inference tables. Built-in controls behave as expected in the examples, while the custom rule initially performs unreliably until a more specific prompt and gpt-5-4-mini evaluator improve triggering without degrading other tests. Inference tables record request outcomes, evaluator calls, latency, statuses, token counts, and shared request\_id values for auditing and refinement.


[Newer posts](https://yomu.fyi/posts/page/126.md) · [Older posts](https://yomu.fyi/posts/page/128.md)