Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Model risk management in 2026: A banker's guide to the revised interagency guidance
The post presents the April 17, 2026 revision to model risk management guidance as a shift toward risk-based, principles-driven oversight, replacing several earlier issuances from the Federal Reserve, FDIC, and OCC. It identifies five operational changes: materiality tiering, end-to-end lifecycle governance, reproducible effective challenge, continuous drift and performance monitoring, and application of MRM principles to GenAI and agentic systems. Its proposed response is a Databricks reference architecture built on Unity Catalog, Delta Lake, Lakeflow Declarative Pipelines, Feature Store, MLflow, Model Registry, Model Serving, and assurance tools. The design makes tiering metadata-driven and turns lineage, validation, approvals, monitoring, documentation, and retirement records into evidence generated during normal model work. The stated conclusion is that a unified substrate can reduce integration and manual evidence work, allowing regulatory changes to become configuration exercises rather than multi-quarter programs.
OpenAI GPT-5.5 + Codex, now available and fully governed on Databricks
Databricks announces native support for OpenAI’s GPT-5.5, making it available for coding workflows with Codex, enterprise agents, document pipelines, and data-driven employee workflows. Access is governed through Unity AI Gateway, which provides permissions and rate limits, configurable guardrails for PII, prompt injection, and content safety, MCP tool-call auditing, failover, and request-level observability for model and Codex interactions. The announcement describes GPT-5.5 use with Genie for natural-language analytics, Agent Bricks Custom Agents for multi-step workflows, and Lakeflow Spark Declarative Pipelines for document-focused GenAI ETL involving summarization, extraction, and classification. GPT-5.5 is available on Databricks across AWS, Azure, and GCP, with entry points including AI Playground, governed Codex endpoints, Agent Bricks, and the Foundation Model API.
Operational databases: How they work and when to use them
Operational databases, or online transaction processing (OLTP) databases, support day-to-day operations by processing real-time CRUD transactions with low latency, high concurrency, and ACID guarantees. The guide contrasts them with data warehouses and OLAP systems, which favor historical analysis, complex aggregations, and read-heavy workloads, while describing ETL, CDC, and operational data stores in the OLTP-to-OLAP pipeline. It argues that traditional OLTP systems can struggle with real-time analytics and AI because of siloed data, structured-data constraints, rigid schemas, scaling limits, and missing governance features. Databricks Lakebase is presented as a hybrid solution using separate storage and compute, serverless Postgres, elastic scaling, branching, shared lakehouse storage, and open formats. The proposed workflow connects existing OLTP systems through CDC or streaming into Delta Lake for fresh SQL, BI, ML, and AI access.
DeepSeek-V4: a million-token context that agents can actually use
DeepSeek V4 is presented as an open model family aimed at long-running agentic workloads, with Pro and Flash MoE checkpoints offering 1M-token context windows. Its central architecture alternates Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA), compressing KV entries by 4x and 128x respectively, while using FP8/BF16 storage and FP4 indexing. The post-training design preserves reasoning across user turns when tool calls are present, introduces a dedicated XML tool-call schema, and relies on DSec, a Rust sandbox platform for RL rollouts. At 1M tokens, Pro uses 27% of V3.2's single-token FLOPs and 10% of its KV cache, while Flash uses 10% and 7%; V4-Pro-Max reports 80.6% on SWE Verified and 0.59 MRCR 8-needle retrieval at 1M.
What's New in pg_clickhouse - JSONB Support, SQL value functions, Streaming, and more
Recent releases of pg_clickhouse, a Postgres foreign data wrapper for ClickHouse, introduce query pushdown enhancements and streaming result handling. Version 0.1.10 adds pushdown for JSONB accessor operators and extraction functions by mapping them to ClickHouse sub-column syntax and toJSONString calls outside SELECT clauses. In version 0.2.0, date and time functions such as CURRENT_DATE, CURRENT_TIMESTAMP, and clock_timestamp() push down to ClickHouse while honoring the active Postgres session time zone and precision settings. To address memory pressure and out-of-memory risks from large foreign queries, the extension adds query result streaming to the HTTP driver with a default batch buffer of approximately 50MB. Container benchmarking against the NYC taxi dataset showed streaming reduced peak memory consumption from over 600 MiB to under 86 MiB.
Databricks partners with OpenAI on GPT-5.5
Databricks announces a partnership with OpenAI around GPT-5.5, described as OpenAI’s strongest frontier model for enterprise agentic work, complex document reasoning, and long-horizon coding agents. The model powers Codex and is presented as able to research, analyze data, create documents and spreadsheets, operate software, use tools, check outputs, recover from ambiguity, and continue through multi-part tasks. Databricks evaluated it on OfficeQA, a benchmark built from 89,000 pages of U.S. Treasury Bulletins that tests document retrieval, table interpretation, and precise calculation. With retrieval handled, GPT-5.5 scored 64.66% versus GPT-5.4’s 57.14%; in the full-agent OfficeQA Pro Agent Harness, it scored 52.63% versus 36.10%, representing reported improvements of roughly 13% and a 46% reduction in errors.
NYC Driver Benefits
Eligible Uber drivers in New York City may receive minimum earnings and several protections established through an agreement involving Uber, the New York Attorney General, and New York City Taxi and Limousine Commission (TLC) rules. For trips beginning in the city, the TLC minimum-pay formula considers time, distance, and driver utilization, while incentives and guarantees count toward earnings but tips, customer-charged fees, and offline promotions do not. As of March 1, 2026, the listed rates are $1.283/mile and $0.681/mile, with different rates possible for wheelchair accessible vehicle (WAV) trips and out-of-city portions. Eligible drivers may also access Black Car Fund benefits, possible unemployment insurance, and paid sick leave, which accrues at one hour per 30 hours worked up to 56 hours yearly and can be redeemed for $18.58 per hour as of that date.
How We Built a Smarter Pickup Experience for Gated Communities
Pickups in gated communities account for up to 30% of Lyft rides in certain markets, yet locked gates and lack of advance instructions historically resulted in higher cancellation rates and extended wait times. To resolve these friction points, Lyft's Mapping team developed an end-to-end mapping and routing framework. The team built an algorithm to generate gate area boundaries, enabling the app to detect gated communities and offer riders explicit choices between waiting inside or meeting outside the gate. Routing services incorporate the gate as an intermediate waypoint to direct drivers along the proper approach path while triggering timely gate access instructions. Following deployment, rides with shared gate instructions exhibited lower cancellation rates from both parties, shorter walking distances to pickup spots, and fewer extended driver wait times.
Guide to BI Reporting and Maximizing Intelligence Effectiveness
Business intelligence (BI) reporting turns data from operational systems and analytical repositories into structured reports, dashboards, and visualizations for organizational decision-making. The guide explains managed and ad hoc reporting, then traces a typical workflow through data sources, ETL, deduplication, normalization, validation, data modeling, visualization, and automated distribution. It compares established platforms such as Microsoft Power BI, Tableau, SAP BusinessObjects, Qlik, and Looker with newer AI-native tools, while noting evaluation criteria including connectivity, scale, governance, usability, and cost. It also addresses dashboard overload, stale semantic models, and specialist bottlenecks, and cites reports of 10x faster query creation from AI-assisted natural-language querying alongside recommendations to start with one decision, one primary source, and three to five KPIs.
Introducing the Sales Agent Blueprint
The Sales Agent Blueprint is a newly announced guide for sales, revenue, and AI transformation leaders seeking to launch and scale AI for inbound sales. It addresses manual inbound workflows in which leads wait in queues, making outcomes depend on team timing and capacity rather than buyer intent, and describes agents that engage buyers, qualify leads, and route high-intent opportunities in real time. The first available track, “Launch it,” covers understanding Sales Agents, building a business case, deciding whether to buy or build, evaluating performance with a five-step framework, planning deployment, and continuously training, testing, deploying, and optimizing the system. A second track, “Scale it,” is coming soon and will focus on redesigning the buyer journey, AI-first systems and ownership models, and pipeline generation and scaling.
National Australia Bank accelerates legacy migrations with Cursor
NAB standardized 6,000 developers on Cursor after evaluating Amazon Q and GitHub Copilot, and is expanding access to more than 10,000 technology employees. The bank uses Cursor across legacy modernization and new development: BizCalc's Silverlight/.NET monolith is being refactored into Java microservices and React, while Assembly-based core-banking programs are being migrated using generated flowcharts and business summaries. Engineers use Ask Mode, Plan Mode, custom NAB CEL skills, Composer, and Opus coding models to document systems, produce requirements and API specifications, parallelize implementation, and test changes. NAB reports that both modernization efforts are progressing three times faster than expected, while a hardware-agnostic payment app was built in under three weeks rather than four months; one team reported a 5–8x velocity improvement.
Announcing the Public Preview of Lakeflow Designer
Databricks announces the Public Preview of Lakeflow Designer, a visual, no-code, AI-native experience for data preparation and analytics built directly into the Databricks platform. It targets analysts, domain experts, and other less technical users with drag-and-drop workflows and natural-language authoring, while representing each transformation as an operator with previews. Genie Code can use Unity Catalog metadata, descriptions, lineage, popularity, and example queries to generate or modify workflows, inspect transformation results, and iterate when needed. Every visual transformation produces production-ready Python code that can be reviewed, versioned in Git, and scheduled through Lakeflow Jobs, while users pay for compute rather than per-user licenses. The preview is available in all workspaces, although an administrator may need to enable Visual data prep in the preview portal.
How to Use Transformers.js in a Chrome Extension
The guide explains how to build a Chrome extension that runs local AI with Transformers.js under Manifest V3, using a browser assistant as the reference architecture. It assigns model orchestration, lifecycle management, tool execution, and conversation state to a background service worker, while a side panel handles chat and a content script handles page extraction and highlighting. The implementation uses Gemma 4 E2B in q4f16 for text generation and all-MiniLM-L6-v2 in fp32 for embeddings, with WebGPU, DynamicCache-based KV caching, explicit download progress, and reusable pipelines. Typed message contracts, model-output parsing, and separate internal and UI transcripts support a deterministic tool loop, while chrome.storage.local, IndexedDB, and background caches place data according to lifecycle and access needs; the guide concludes that separating these runtimes keeps the UI responsive and avoids duplicate model loads.
Do you still need Elasticsearch for log analytics? ClickHouse says no.
ClickHouse introduced a redesigned full-text search powered by inverted indexes that feeds matching documents directly into its vectorized analytical engine for observability and log analytics workloads. To evaluate this capability, the open-source versions of ClickHouse and Elasticsearch were benchmarked on single-node AWS EC2 instances across synthetic OpenTelemetry log datasets scaling up to 50 billion rows. The benchmark evaluated nine representative query patterns combining text search with filtering, grouping, counting, and time-series aggregations under both cold and hot cache conditions. ClickHouse achieved a roughly fivefold smaller storage footprint on disk while running cold queries four to six times faster and hot queries approximately 1.7 to 2.6 times faster than Elasticsearch. Furthermore, ClickHouse ingested the 50-billion-row dataset in under four hours out of the box, whereas the tuned Elasticsearch setup required approximately five days.
https://www.notion.com/en-gb/blog/topic/podcast
The page is a Notion blog topic archive for “Pioneers,” identified by the slug “podcast” and described as conversations with the minds that shaped computing. It presents the blog’s navigation, including sections for Notion HQ, For Teams, Inspiration, Pioneers, and Tech, followed by a latest-posts feed with titles, categories, authors, and excerpts. The visible entries cover subjects including Notion’s Developer Platform, Custom Agents, hosted MCP server, Notion Calendar, multi-region data systems, schema-change detection, and page design. The supplied page data records 56 total items and identifies the canonical path as /blog/topic/podcast.md, but the visible excerpt does not provide individual podcast episode details.
Are LLM agents good at join order optimization?
Join ordering is difficult because relational optimizers choose among exponentially many plans and depend on potentially inaccurate cardinality estimates. Databricks and UPenn researchers built an LLM agent that uses structured outputs to generate valid join reorderings and executes candidates through a tool returning runtime and intermediate subplan sizes. It can test up to 50 iterations, while the evaluation tested 15 rollouts per query across 113 Join Order Benchmark queries after duplicating each dataset row ten times. With a frontier model, the best tested orders improved geometric-mean latency by 1.288 times and reduced P90 latency by 41% versus the standard Databricks optimizer. A five-way example links its gain to LIKE predicates that are exceptionally difficult for cardinality estimators.
Introducing HoosierShare, Indiana University’s innovative program with Uber
Indiana University Bloomington has partnered with Uber to launch HoosierShare, a 24/7 shared-ride option for verified students, faculty, and staff traveling around Bloomington and campus. Powered by Uber’s Closed Loop Share feature, the program matches IU riders heading in the same direction, permits up to two other individual riders, and can cost up to 20% less than standard UberX fares. Enrollment requires an Uber account, IU-credential verification, and use of the designated IU Ride service zone, where every trip must start and end; HoosierShare allows one person per booking. The program is intended to expand transportation access and support IU sustainability goals, while the first 10,000 enrollees receive $5 off their first trip and shared rides are designed to add no more than eight minutes on average.
How to Influence LLM Responses: A Guide to AI Visibility
AI tools increasingly answer consumer questions directly, reducing click-throughs while making remaining referrals more likely to convert, according to the Post. Brands therefore need to influence the broader information environment that shapes LLM recommendations, not merely optimize the URLs shown as citations. LLMs draw on owned content, earned media, and social conversations, including crawlable trade publications, industry blogs, Reddit, forums, and podcast transcripts. A telecommunications brand illustrates the owned-content route: after identifying real troubleshooting questions and publishing targeted answers, it reduced call center contact volume by 25% and improved issue resolution. The guidance also stresses concrete claims, authentic participation, and monitoring outdated or inaccurate narratives to build fresher context around a brand.
How conversational analytics removes the BI bottleneck
The interview presents conversational analytics as a way to move beyond fixed dashboards and turn governed data into decisions and workflows. Databricks’ Genie lets nontechnical users ask open-ended questions in plain language across Lakebase, Lakehouse, and federated sources, while Unity Catalog applies shared definitions and access controls. Lakebase provides transactional storage, instant provisioning, environment forking without copying underlying data, and scale described as billions or trillions of records; examples include real-time matchmaking, routing, and purchase decisions. The discussion stresses that human oversight, refreshed data, business context, and controls against hallucination remain necessary, citing Fox Sports’ chatbot deployment as an example. It concludes that organizations combining trusted semantics with operational data systems can make faster decisions and automate work, while the competitive gap may compound for slower adopters.
How to transform document activation workflows with Genie and Agent Bricks
Enterprise document workflows often require people to read PDFs, copy fields into spreadsheets, and re-enter data into business systems, creating delays, error risk, and weak auditability. The proposed alternative combines AI/BI Genie for natural-language queries over Delta tables, Agent Bricks for production agents, and Unity Catalog for permissions, lineage, and traceability. A five-phase workflow extracts fields into Lakeflow Bronze, Silver, and Gold layers, then supports structured queries, clause-level RAG answers, multi-agent routing, and actions through MCP servers. Examples include syncing validated rights data to ERP or CRM systems, updating billing entitlements, and triggering workflows from detected obligations. The approach targets media, advertising, ad tech, and telecommunications, where customers see improvements including faster month-end close, recovered revenue, reduced leakage, and lower operational risk.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.