---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [AiChemy: Next](https://yomu.fyi/post/aichemy-next.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Yen Low, Sean Zhang
- Published: Sep 18, 2026
- Type: Tutorial

AiChemy is a multi-agent assistant for cross-disciplinary research that combines external MCP servers including OpenTargets, PubChem, and PubMed with Databricks-managed chemical libraries. It can identify disease targets and drug candidates, retrieve chemical and pharmacokinetic properties, assess safety and toxicity, and attach findings to verifiable data sources. Its guided workflow covers disease-to-target-to-drug discovery and literature validation, while a vector search over a 250,000-molecule ZINC15 index uses 1024-bit ECFP embeddings to find compounds similar to Elacestrant. To build a custom supervisor, users connect public MCP servers, transform structured drug data into a Genie space, and index unstructured chemical data for similarity search. They can assemble it with no-code Agent Bricks or deploy a Langgraph supervisor with Lakebase, MLflow AgentServer, and a React interface; invocations are logged and traced through MLflow using OpenTelemetry standards.


### [How MakeMyTrip achieved millisecond personalization at scale with Databricks](https://yomu.fyi/post/how-makemytrip-achieved-millisecond-personalization-at-scale-with-data.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Sitesh Sharma, Aditya Kumar, Navneeth Nair
- Published: Sep 18, 2026
- Type: Problem & solution

MakeMyTrip needed sub-second personalization for “last-searched” hotels across its consumer and corporate travel businesses, serving millions of daily users. Spark Structured Streaming’s micro-batch mode consistently delivered one to two seconds, while adopting Flink would have added a second engine, duplicated logic and increased operational overhead. The team instead adopted Databricks Real-Time Mode (RTM), using continuous data flow, concurrent pipeline scheduling and streaming shuffle to process events with millisecond-level latency on Spark. Its unified pipeline merges B2C and B2B Kafka streams, enriches events with each user’s “Last N” hotels from Aerospike, and writes results to Redis for serving in under 50ms. After tuning partitions, backpressure and rate limits, the production deployment reduced P50 latency from approximately 1.23 seconds to 44ms, P99 latency from over one minute to about 500ms, and increased click-through rate by 7%.


### [How FSIs eliminate silos between clients, operations, and finance](https://yomu.fyi/post/how-fsis-eliminate-silos-between-clients-operations-and-finance.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Anindita Mahapatra, Emily Piekarski, Shirly Wang
- Published: Sep 18, 2026
- Type: Problem & solution

Financial institutions often split portfolio, operations, actuarial, and finance work across separate systems, even when teams depend on the same portfolios, positions, liabilities, and transactions. Databricks presents a unified, governed platform combining Databricks SQL, Unity Catalog Metric Views, Lakebase, Genie, Databricks Apps, and Lakeflow Designer to connect analysis with operational action. A worked workflow follows an actuary's duration analysis, a portfolio manager's scenario modeling and mandate changes, operations' IBOR/ABOR reconciliation, and finance's approval of adjustments and ledger entries. Governance features including row-level security, column masking, RBAC, ABAC, and shared business semantics keep access and terminology consistent, while Lakebase records corrections in auditable, timestamped tables. The result described is a closed-loop process in which strategy, risk, and accounting use the same governed data rather than reconciling separate systems.


### [How Zalando built a unified data foundation for AI and analytics on Databricks](https://yomu.fyi/post/how-zalando-built-a-unified-data-foundation-for-ai-and-analytics-on-da.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Fabian Halkivaha, Mukrram Ur Rahman, Maria Vedenina, Timur Yüre
- Published: Sep 18, 2026
- Type: Explainer

Zalando describes how it is building a unified data foundation for analytics and AI across a microservices-driven ecosystem serving more than 50 million active customers and 7,000 brands and partners. Its architecture combines private domain-owned catalogs for autonomous development with a centrally governed shared catalog that exposes cross-team data through Dynamic Views, enabling identity-based policies, GDPR checks, column classification, and auditability. Databricks Metric Views provide a governed semantic layer where business logic and KPI definitions are centralized and reusable across SQL, dashboards, notebooks, and AI agents. Genie uses those metrics plus enriched metadata to answer natural-language questions more reliably, although poorly curated or unaggregated data can increase costs; Zalando plans broader enterprise adoption through curated spaces and additional orchestration.


### [Powering clinical research innovation: How TriNetX uses Databricks to accelerate drug development](https://yomu.fyi/post/powering-clinical-research-innovation-how-trinetx-uses-databricks-to-a.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Jeff Warnick
- Published: Sep 18, 2026
- Type: Explainer

TriNetX operates a federated real-world health data network spanning more than 230 healthcare organizations and needed infrastructure to make growing volumes of RWD easier to use for clinical research. Databricks now provides the centralized lakehouse architecture, consolidating electronic health record data while supporting custom datasets, consulting, ML models, proprietary algorithms, APIs and self-service access. TriNetX is extending this foundation with a beta Query Assistant for natural-language analysis and a Support Assistant prototype built with Databricks’ Agent Bricks, requiring no programming expertise for the former. In 2025, pharmaceutical clients reduced protocol amendments by up to 50%, while an AI-enhanced site-identification effort achieved a 63% site acceptance rate and nine-day average response time. Other models suggest enrollment conversion could rise from 33% to 85%, with expanded APIs and Genie exploration planned for 2026.


### [What is AIOps?](https://yomu.fyi/post/what-is-aiops.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Sep 17, 2026
- Type: Explainer

AIOps applies AI and machine learning to IT operations to analyze logs, metrics, traces, events, alerts, and related system data. It detects anomalies, correlates events, uses topology and dependencies to identify likely root causes, and supports guided or automated responses. The guide positions AIOps between observability and action, reducing noise and adding context without replacing observability, DevOps, or human judgment. Its workflow includes data ingestion, normalization, anomaly detection, event correlation, root-cause analysis, automation and orchestration, and collaboration workflows, with human-in-the-loop review recommended for higher-risk production changes. It contrasts domain-centric and domain-agnostic approaches and recommends evaluating coverage, depth, integrations, setup effort, and organizational fit before expanding automation.


### [The official ClickHouse provider for Apache Airflow is now available](https://yomu.fyi/post/the-official-clickhouse-provider-for-apache-airflow-is-now-available.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Sep 17, 2026
- Type: Announcement

Apache Airflow users can now use an upstream ClickHouse provider, offering an officially maintained alternative to community plugins or custom integration code. The provider, apache-airflow-providers-clickhousedb, uses ClickHouse Connect over HTTP(S), integrates with Airflow’s common SQL framework, and supplies ClickHouseHook for bulk inserts, streaming, and client-specific operations. The post documents installation, connection setup, standard SQLExecuteQueryOperator DAGs, bulk insertion, and local or managed deployment through Astronomer, including TLS settings for ClickHouse Cloud. Production examples include Astronomer’s Astro Observe, Chartmetric’s 15-million-row playlist pipeline every five minutes, and ClickHouse’s warehouse, which schedules 76 DAGs across more than 40 sources and moves about 6 billion rows daily. The provider is available now, with future capabilities guided by community requests.


### [How Equinix cut operational overhead with a shared services architecture on Amazon EKS](https://yomu.fyi/post/how-equinix-cut-operational-overhead-with-a-shared-services-architectu.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Chhavi Kaushik
- Published: Sep 17, 2026
- Type: Problem & solution

Equinix replaced its self-managed Kubernetes environment on Amazon EKS to address operational sprawl caused by independently managed clusters and inconsistent infrastructure practices. The new shared services architecture assigns platform ownership to a cloud operations team while application teams focus on business services. It uses separate Workloads, Platform, and Network accounts: workloads run across two Availability Zones with ALB, Gateway API, Cilium, and Hubble, while shared data services, GitHub Runners, and platform services reside in the Platform account. The Network account provides cross-account and multi-Region connectivity through AWS Transit Gateway, Route 53 Resolver, and Direct Connect. Equinix reports 40% lower operational overhead, 4x higher deployment frequency, and adoption of a unified architecture across multiple business organizations.


### [The Web Search Your Agent Inherited Isn't Good Enough](https://yomu.fyi/post/the-web-search-your-agent-inherited-isn-t-good-enough.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Charlie Klein, Bryan Smith
- Published: Sep 17, 2026
- Type: Problem & solution

An agent that keeps a company’s market view current must combine Databricks-hosted internal account records with external web signals such as funding rounds, leadership changes, product launches, and hiring surges. The initial implementation duplicates enrichment logic across Claude Code, Codex, and a direct model API, with inconsistent search results and no shared cost, governance, or audit trail. Omnigent consolidates the agent definition, model, tools, policies, limits, and model-call governance above those harnesses, while letting the engineer select a common search provider. Nimble’s Search API supplies live web data, while its Web Search Agents orchestrate multi-source retrieval and extraction, cross-check sources, return citations, and adapt to difficult sites and recurring retrieval paths. In Nimble’s testing, adding its search increased LLM benchmark accuracy from 46% to 71% and halved web search costs; the proposed Databricks setup then connects internal records and external context under one governance surface.


### [Introducing the Life Sciences Verification Program](https://yomu.fyi/post/introducing-the-life-sciences-verification-program.md)
- Company: [Anthropic](https://yomu.fyi/company/anthropic.md)
- Published: Sep 17, 2026
- Type: Announcement

The Life Sciences Verification Program (LSVP) opens beta access to Mythos, Opus, and Sonnet models for verified life science teams whose work may be blocked by generally available Fable models. Applicants undergo review of research credentials, security standards, and ethical oversight, then may receive annual Standard Use grants or six-month, project-specific High-risk Use grants. Standard Use employs more permissive biology classifiers across research, clinical, manufacturing, and other workflows, while High-risk Use removes life-science safeguards for specifically vetted projects; cyber safeguards remain. The program shifts enforcement from request-level blocking to offline monitoring of behavioral patterns, requiring 30-day retention of flagged-activity data, compartmentalized from training and life sciences research teams. Access is available through API, Claude Science, Claude.ai, Claude Code, and Enterprise or Team plans, but not individual plans, third-party platforms, or BAA-enabled organizations during beta.


### [The guest journey, updated in real time: extending Airbnb’s sequence recommender with Chronon](https://yomu.fyi/post/the-guest-journey-updated-in-real-time-extending-airbnb-s-sequence-rec.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Pengyu Hou
- Published: Sep 17, 2026
- Type: Problem & solution

Airbnb’s guest sequence recommender previously refreshed Transformer-based embeddings in a daily batch, leaving activity from earlier the same day unavailable until the following run and creating up to nearly two days of staleness. To close this gap, Airbnb added Push Mode, which emits notifications when streaming feature values commit, and Near-real-time Model Transform, which invokes the deployed sequence encoder inside that streaming pipeline. Each page view or search is merged with the guest’s long-term and short-term sequence data, triggers inference, and writes a fresh embedding to the same low-latency store used by batch serving. The resulting system typically updates embeddings in under a minute, often within 10–30 seconds, improved offline NDCG by 1.67%, and increased uncancelled bookings by approximately one-third of a percent in online A/B tests.


### [A skills library for every agent](https://yomu.fyi/post/a-skills-library-for-every-agent.md)
- Company: [Notion](https://yomu.fyi/company/notion.md)
- Author: Geoffrey Litt
- Published: Sep 17, 2026
- Type: Announcement

The post introduces Notion’s skills library and Skills API, aimed at helping teams share organizational knowledge across AI agents and workflows. The library is designed for nontechnical and technical users, with collaborative editing, database organization, supporting files, permissions, suggested edits, version history, usage analytics, and controls over skill distribution. The API provides endpoints for loading skills in spec-compliant file formats, with examples including synchronization to GitHub, installation through Vercel’s Skills CLI, and uploads or edits through MCP tools. Notion says early customers and partners are already using the API across agents and tools, while an open-source starter kit supports Notion-to-GitHub synchronization. Vercel’s Skills CLI can install individual Notion skill pages or workspace plugins, using standards-compliant files without handling Notion-specific formats.


### [Building cloud-native PACS on AWS](https://yomu.fyi/post/building-cloud-native-pacs-on-aws.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: ManojKumar MV
- Published: Sep 17, 2026
- Type: Explainer

Multi-hospital networks face rising imaging volumes, siloed on-premises PACS, radiologist routing bottlenecks, and storage that must expand for 7 to 10 years of retention. The proposed pattern uses a hub-and-spoke hybrid architecture: local PACS instances preserve LAN-speed reading, while a centralized AWS archive provides cross-facility discovery, disaster recovery, and lifecycle-managed storage. Modalities send DICOM through C-STORE to local VNA servers, which normalize and compress objects; asynchronous replication sends images to Amazon S3, while Amazon Aurora PostgreSQL stores metadata and routing information. The AWS deployment spans two Availability Zones, places PACS services behind Network Load Balancers, and includes CloudFront and AWS WAF. Storage guidance distinguishes S3 Intelligent-Tiering, Glacier Instant Retrieval, and Glacier Deep Archive according to access frequency and retention needs.


### [How DHI Group accelerates generative AI workloads from idea to production using hackathons](https://yomu.fyi/post/how-dhi-group-accelerates-generative-ai-workloads-from-idea-to-product.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Umesh Kalaspurkar
- Published: Sep 17, 2026
- Type: Explainer

DHI Group needed to move generative AI efforts beyond proofs of concept because its traditional software development lifecycle took months, while it sought faster delivery, technical validation, and broader AI literacy. AWS and DHI used a four-phase Hackathon Acceleration Package—preparation, enablement, a three-day build event, and a path to production—with workshops on Amazon Bedrock AgentCore, AI-DLC, and Kiro. Twenty participants formed three teams and produced prototypes for employer analytics, intelligent candidate matching, and a unified ClearanceJobs–AgileATS marketplace; DHI leadership committed to advancing all three toward production. The winning marketplace uses an MCP server, Amazon Bedrock AgentCore, Lambda functions, session memory, and Claude 3.5 Haiku to let recruiters search candidates and enrich profiles through natural-language requests. The post concludes that time-boxed, cross-functional hackathons can compress ideation, architecture, prototyping, executive alignment, and production planning when production criteria and preparation are established upfront.


### [Postgres week in the Netherlands: PGDay Lowlands & Percona Live 2026](https://yomu.fyi/post/postgres-week-in-the-netherlands-pgday-lowlands-percona-live-2026.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Sep 17, 2026
- Type: Explainer

An account of three days in the Netherlands covers talks at PGDay Lowlands in Utrecht and Percona Live 2026 in Amsterdam, including a five-minute presentation on ClickHouse integrations and a PostgreSQL 19 monitoring session. pg\_clickhouse uses a foreign data wrapper to expose ClickHouse tables as PostgreSQL foreign tables, allowing existing queries, ORMs, and dashboards to continue working while pushable work runs in ClickHouse; 15 of 22 TPC-H queries at scale factor 1 are fully pushed down. pg\_stat\_ch captures per-query execution events in a shared-memory ring buffer and sends batches to ClickHouse, dropping events rather than blocking PostgreSQL when ClickHouse is unavailable. The monitoring talk covered PostgreSQL 19 logging, WAL and I/O statistics, WAIT FOR, new system views, and multixact and wraparound changes. The post also reports talks on optimizer hints and plan stabilization, and says both conferences will return in 2027.


### [September 2026 newsletter](https://yomu.fyi/post/september-2026-newsletter.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Sep 17, 2026
- Type: Announcement

The September 2026 ClickHouse newsletter gathers product updates, community contributions, technical articles, and upcoming events. Its main release coverage describes ClickHouse 26.8, including custom HTTP handlers, dynamic query filtering, pipelined SQL through the |> operator, current-user query logging, a URL database engine, and Japanese and Chinese text-index tokenizers. Other featured developments include On-Demand Compute in private preview with a cost-based optimizer and distributed query execution, PromQL and a TimeSeries engine in private preview, AI Functions, and Postgres integrations such as WalShadow and Managed Postgres on Google Cloud. The issue also reports CostBench’s first end-to-end performance-per-dollar results, explains replication-queue diagnosis, notes 57 breaking changes since 26.3, and lists training, webinars, roadshows, and other events.


### [From hours to minutes: Optimizing Red Hat Developer Hub performance testing with immutable LDAP images](https://yomu.fyi/post/from-hours-to-minutes-optimizing-red-hat-developer-hub-performance-tes.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Shashank Kestwal
- Published: Sep 17, 2026
- Type: Benchmark

Red Hat Developer Hub performance testing became slow and unreliable as runtime generation of large user and group catalogs stretched environment setup and introduced synchronization failures. Previously, the framework created entries through serialized Keycloak Admin REST API operations, then waited for paginated synchronization into the Developer Hub database; concurrency could not eliminate timeouts, rate limits, or incomplete catalogs. The replacement uses immutable, pre-built LDAP container images selected by an RBAC\_POLICY-USER\_COUNTu-GROUP\_COUNTg convention, with slapadd loading seed data directly into offline LMDB files in a multi-stage build. LDAP starts with 10,000-plus indexed users, while Keycloak serves as an authentication gateway and Developer Hub waits for its initial catalog sync. Reported population time fell from 22-plus hours to 2 minutes 8 seconds for 10,000 users and 150,000 groups, while the immutable approach made tests deterministic, reduced flaky failures, stabilized baselines, and enabled larger reproducible scenarios.


### [Catching poor LLM performance and accuracy before deployment](https://yomu.fyi/post/catching-poor-llm-performance-and-accuracy-before-deployment.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Christopher Miyai
- Published: Sep 17, 2026
- Type: Benchmark

Rapid changes in vLLM and the path from upstream releases to Red Hat AI Inference images can introduce performance and accuracy regressions that ordinary CI/CD tests miss. The midstream regression detector benchmarks upstream and release-candidate builds across hardware variants, using GuideLLM at multiple concurrency levels for performance and GSM8K for accuracy. It stores nightly upstream results, compares them with midstream benchmarks against threshold deltas, and feeds confirmed failures into fixes and rebuilds before release. To reduce false positives from limited samples, the workflow adds Wasserstein distance as a secondary check of whole request-response distributions rather than relying only on medians or means. The system also connects benchmark runs with prior JIRA regressions through a custom Claude skill that suggests possible fixes and assesses whether a flagged change is genuine.


### [Build a unified CI/CD control plane with Red Hat Developer Hub](https://yomu.fyi/post/build-a-unified-ci-cd-control-plane-with-red-hat-developer-hub.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Rishabh Singh
- Published: Sep 17, 2026
- Type: Tutorial

Red Hat Developer Hub brings OpenShift, Argo CD, Tekton, and Quay workflows into a unified developer portal, addressing context switching across separate consoles. The guide targets platform engineers and covers installing the Operator, creating GitHub and cluster credentials, configuring application and dynamic-plugin settings, defining RBAC policies, and deploying a Backstage custom resource. It uses a Kubernetes service-account token for access to pods, deployments, and pipeline runs, while Quay integration exposes image tags, vulnerabilities, and manifest details. After granting read access in monitored namespaces, the resulting portal can display pipeline runs, deployment health, image information, and Argo CD status in one view. The source says the setup is a proof-of-concept foundation, with a follow-up planned for software templates and developer self-service.


### [Building a RAG Pipeline for Semantic Code Search: A Developer Diary and Field Notes](https://yomu.fyi/post/building-a-rag-pipeline-for-semantic-code-search-a-developer-diary-and.md)
- Company: [Jetbrains](https://yomu.fyi/company/jetbrains.md)
- Author: Adam Malek
- Published: Sep 17, 2026
- Type: Explainer

JetBrains Context is presented as a production RAG pipeline for semantic code search, designed to give coding agents precise, citable evidence from repositories instead of relying on keyword tools such as grep. This first installment covers parsing, structure-aware chunking, and vectorization, with the goal of retrieving meaningful functions, symbols, or snippets rather than whole files or insignificant lines. Language parsers produce syntax nodes, and the algorithm retains the largest units that fit, splits oversized leaf nodes when necessary, groups small adjacent units, and keeps documentation, annotations, modifiers, and closing syntax with their constructs. JetBrains Context supports this approach for Kotlin, Java, Python, JavaScript, TypeScript, C#, PHP, Go, and Rust; other languages use line-based splitting. Vectors are generated with an open-weight model on company-operated GPUs, while indexed records store coordinates rather than source code, and the post reports that this setup outperformed hosted embedding APIs on internal code-retrieval benchmarks.


[Newer posts](https://yomu.fyi/posts/page/41.md) · [Older posts](https://yomu.fyi/posts/page/43.md)