Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,668 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Clinical operations intelligence belongs on the Lakehouse
The Site Feasibility Workbench addresses fragmented clinical operations data by placing a site-selection decision-support application, its models, and data on Databricks. Released as a fully open-source Databricks App, it combines a FastAPI backend, React frontend, Unity Catalog-governed data, Lakebase operational state, and AI/BI Genie within the workspace. TA-segmented LightGBM models use the organization’s CTMS, EDC, and IRT history alongside real-world evidence, patient access, qualification, Open Payments, and protocol-execution signals to rank sites. Each prediction stores SHAP attributions in a governed Unity Catalog Delta table, while MLflow tracks training runs and model versions; saved shortlists persist to Lakebase and Genie answers governed-data questions. The six-step workflow supports protocol selection through final shortlisting, including diversity considerations; the release is a decision-support layer rather than a source of record, and deployment reportedly takes about 30 minutes before security review and validation.
The ultimate guide to knowledge management for your Sales Agent
AI-powered inbound sales depends on knowledge management: accurate, current information about products, pricing, plans, qualification, and routing gives a Sales Agent the context to answer questions and guide buyers. The guide describes knowledge management as creating, organizing, sharing, and maintaining business knowledge, while emphasizing content such as product FAQs, pricing overviews, competitor battlecards, case studies, and internal sales materials. It recommends finding gaps through recurring questions, buyer objections, search data, and disengaged conversations; then writing machine-friendly content with simple language, consistent terminology, explanatory context, scannable formatting, and text accompanying images or video. Connecting the Agent to CRM and enrichment systems keeps data and context connected, while ongoing reviews help content evolve from conversation insights and reduce repetitive work for sales teams. It presents knowledge management as a continuous sales function whose cumulative improvements can support faster responses, better qualification, and more focused human selling.
Introducing Notion’s Developer Platform
Notion introduces a Developer Platform with Workers, data synchronization, custom agent tools, external agents and a CLI for extending its workspace beyond built-in capabilities. As teams connect agents to more workflows, the platform addresses gaps involving external data, custom logic and agents that cannot otherwise participate in Notion. Workers run custom code in a hosted secure sandbox, enabling database sync, agent tools and webhook triggers without user-managed servers; the CLI supports authentication, Notion actions and Worker deployment from a terminal or IDE. External Agents can work natively alongside teams and Custom Agents, while the External Agent API supports bringing internally built agents into the workspace. The CLI is available on all plans, Workers are in public beta for Business and Enterprise plans, and External Agents are offered through a private-beta waitlist.
ClickStack SQL Charting and Alerting
ClickStack introduced SQL-based visualizations and SQL-based alerting to allow users to build charts and alerts using arbitrary ClickHouse SQL queries. Traditional query builders trade analytical flexibility for simplicity, preventing teams from easily expressing multi-stage computations like rolling baselines, anomaly detection, or dynamic SLO calculations. To maintain interactivity within dashboards, the system supports ClickHouse query parameters and macros that inject active time ranges, interval sizes, dashboard filters, and source tables. Furthermore, SQL-based alerting evaluates the numeric columns output by these queries, such as returning binary flags when values exceed statistical boundaries. By shifting computational logic directly into SQL, teams can implement complex operational checks without relying on static external thresholds.
ABAC row filtering and column masking policies, governed tags, and data classification are now generally available in Unity Catalog
Unity Catalog now generally offers three complementary data-governance capabilities: Attribute-Based Access Control (ABAC) policies for row filtering and column masking, Governed Tags, and agentic Data Classification. They address per-object access rules, coordination gaps, and manual detection by letting governance teams define tag-based policies once, automatically classify sensitive data, and protect matching objects across catalogs and schemas. Governed tags provide an account-level vocabulary inherited across catalogs, schemas, tables, and columns, while ABAC applies row filters and column masks using tag-based conditions. Classification uses built-in compliance classifiers, custom classifiers, metadata, pattern recognition, and large language models, with human-in-the-loop validation and false-positive exclusions. General availability adds 10x larger policy limits, support for 10,000+ policies per metastore, lifecycle management through SQL, APIs, UI, and Terraform, expanded compliance coverage, and custom classifiers in beta.
Development environments for your cloud agents
Cloud agents need development environments with repositories, dependencies, credentials, and build-system access to test, query services, and verify engineering work rather than merely write code. The release adds reusable multi-repo environments, allowing agents to work across microservices codebases and deliver, test, and verify changes spanning repositories. It also improves Dockerfile-based configuration with build secrets scoped to build steps and layer caching that makes cache hits run 70% faster. Cursor can inspect repositories, determine required tools and dependencies, and generate editable Dockerfile configurations, while its setup flow checks credentials and validates environments; this generator is in private beta. Governance features include environment-specific version history, rollback controls, audit logs, egress allowlists, and isolated secrets, while failed configurations fall back to a base image and work continues.
Are you winning in AI search? Do this self-assessment.
AI search is reshaping consumer discovery, but the post says most brands do not know what AI platforms say about them. It presents a four-step assessment: choose one country, define a focused category, identify roughly 100 consumer questions, and test whether the brand appears in answers from ChatGPT, Gemini, Perplexity, and Google AI Overviews. Search Intelligence platforms track queries, mentions, recommendations, citations, and content gaps across search and AI tools. The post recommends answering real queries with dedicated content, earning coverage from accessible independent sources, participating helpfully in online conversations, and monitoring AI answers monthly because they change. Its examples report a 25% reduction in Orange call-center contacts, 50% lower Campari costs, and Aromazon reaching €40 million in annual revenue.
App-based Worker Notice of Rights (Seattle)
Uber’s Notice of Rights describes protections and information available to app-based workers using its platform for delivery work in Seattle, Washington, under Seattle Bill CB 120294 and Seattle Municipal Code Chapter 8.37. It lists guaranteed minimums of $0.47 per engaged minute, $0.80 per engaged mile, and $5.34 per offer, along with required information before accepting an offer. Workers may decide when to be available and whether to accept, reject, or cancel delivery offers, and are protected from retaliation for exercising covered rights. The notice also covers tip and fraudulent-use policies, cancellation reasons, complaint or civil-action rights, and access for up to three years to information required under Sections 8.37.070(B) and (C); Uber directs workers to Customer Support for those records.
Full-screen Tabs and Compact Chats
This release introduces quality-of-life improvements to the Agents Window, centered on full-screen work areas and more compact agent conversations. Full screen expands files, changes, canvases, PRs, browsers, or terminals across the right panel, replaces chat with a floating prompt bar, and is available through the panel control, command palette, or Cmd/Ctrl+Shift+M. Compact chats offer Compact, Balanced, and Detailed tool-call density settings, ranging from concise results with minimal traces to near-complete step-by-step context. The release also improves PR review visibility, scrolling, streaming, context persistence, MCP authentication, model selection, search, and task status, while fixing reliability issues across PRs, background and cloud agents, MCP/OAuth, shortcuts, hooks, model routing, webhooks, and mirrored repositories.
Development environments for cloud agents
Cloud agents need a laptop-like development environment with cloned repositories, installed dependencies, credentials, and build-system access to complete engineering tasks end-to-end. This release adds configurable multi-repo environments, Dockerfile-based configuration, and tools that let Cursor set up and maintain those environments. Build secrets remain scoped to build steps, while improved layer caching makes cache-hit builds run 70% faster; Cursor also asks setup questions, flags missing credentials, validates configuration, and falls back to a base image with warnings when setup fails. Version histories, admin-controlled rollback, audit logs, and environment-scoped egress and secrets provide governance and isolation across environments for teams running parallelized agents.
Fast Copy-On-Write within Apache Parquet for Data Lakehouse ACID Upserts
Large-scale lakehouse upserts can become slow and resource-intensive, especially in Apache Hudi copy-on-write mode, where affected files are read, modified, and rewritten in full. The approach adds a Parquet row-level secondary index mapping records to files and row IDs, allowing updates to locate the specific data pages containing changed records. During copy-on-write, related pages are updated while unrelated pages are copied as byte buffers, avoiding decompression, re-encoding, and record reassembly for skipped data. Benchmarks using TPC-DS sales data, Spark jobs with matching vCore and memory settings, and update rates from 5% to 50% found significantly faster speeds than traditional approaches, including Delta Lake’s default configuration. The authors plan to integrate the row-level index and fast copy-on-write into Apache Hudi for use cases such as incremental ingestion, data deletion, and CDC.
How CFOs in consulting can recover margin with Databricks
Consulting firms are facing margin pressure despite growing deal pipelines, with 2024 EBITDA margins at 9.8%, utilization at 68.9%, and revenue growth at 4.6%. The post attributes this to fragmented finance workflows spanning Salesforce, SAP, Workday, Concur, and spreadsheets, plus inconsistent metrics and manual reconciliation. It proposes using Databricks to create a governed data layer with Lakeflow, Lakehouse, and Unity Catalog, then adding Genie for natural-language analysis and finance apps that trigger workflows and write approvals back in real time. A global consulting firm reportedly cut cash forecasting cycles by 3–5 days and reduced reporting-cycle FTE hours by 80%, while Databricks’ own close fell from 15 days to 8. The proposed rollout spans 10–12 weeks and begins with one tightly scoped workflow such as revenue close, unbilled WIP, T&E leakage, or practice-margin analysis.
Today Intercom becomes Fin
Intercom is changing its company name to Fin while retaining Intercom for its customer service software platform, including the recently launched Intercom 2 rebuild. The company says all 1,400 employees are now employed by Fin, named after its leading customer agent platform, which it says is about to become the largest part of the business. The change follows three years of promoting the service agent category under Fin, often with Intercom in the background, and reflects a belief that established brands carry baggage when repositioning. The CEO argues that newcomers benefited from having no prior market position to explain, despite Intercom’s claim of superior technology, and presents the renaming as part of a broader transformation spanning culture, pricing, products, and investment priorities.
An Engineer’s Guide to Better AI Skills: Implementing a Testing Process to Optimize Agent…
Software engineers deploying AI agents often encounter unreliable skill invocation for repository-specific architectures, such as Pinterest's iOS framework. To quantify and resolve this issue, a Bash test harness piped positive and negative prompts into Pin-agent and Claude Code while parsing streamed JSON logs for execution signatures. Initial vanilla benchmarks yielded baseline accuracies of 73% for GPT 5.2-codex and 62% for Opus 4.5 Claude. Optimizations such as expanding YAML frontmatter descriptions with architectural context, utilizing capitalized imperative directives, and adding reference tables in AGENTS.md raised invocation rates. Both agents reliably loaded skills when prompts contained explicit invocations, demonstrating that empirical testing paired with descriptive prompting stabilizes agent skill adoption.
We’ve improved Destination Mode to get you better matches
Destination Mode has been updated to prioritize trip requests that move drivers toward a chosen destination while accounting for added time and distance. Drivers can choose Flexible route, which widens the search area for more opportunities but may lengthen arrival, or Faster route, which limits requests to trips directly on route for potentially quicker arrival but fewer offers. To use it, drivers search for a place in Driver App, enable Destination Mode, select a route, and accept trips. Destination Mode can be used twice daily, with a session starting on an Exclusive trip request; drivers can decline or cancel up to two Exclusive offers on Faster or three on Flexible before the session ends, and those offers still affect Acceptance Rate. Delivery requests remain unfiltered, and the mode cannot be enabled at airports.
ClickHouse Cloud: Fast, Updatable Lookups with the Join Table Engine
ClickHouse Cloud provides an in-memory Join table engine for precomputed hash-map lookups and joins, aimed at enriching append-only fact data with dimensions that change occasionally. With Join(ANY, LEFT, key), inserting another row for a key acts as an upsert, while joinGet and matching JOIN clauses use the in-memory structure. Open-source Join tables are not distributed and persist each INSERT batch as separate .bin files without automatic compaction, but ClickHouse Cloud transparently uses SharedJoin with MergeTree for ALL or ReplacingMergeTree for ANY. In the demonstrated enrichment query, duplicate inserts for id 1 leave value 42 visible, while the underlying table retains duplicates until merging; the in-memory load applies FINAL and always selects the latest entry, making ANY LEFT suitable for frequently updated dimensions.
Lessons from Building a First-Pass AI PRD Reviewer at Uber
Uber built an AI-powered PRD Evaluator to help product managers strengthen drafts before they enter structured checkpoint and approval forums. Starting with a PRD, it gathers linked documents, decks, meeting notes, prior experiments, cross-functional artifacts, and Uber-specific context, then classifies the proposal to calibrate review depth. It assesses opportunity and hypothesis, product scope, user experience and impact, and metric and data rigor, producing a launch-readiness scorecard with prioritized gaps, evidence, replacement-text suggestions, and a “start here” fix. Early internal use by dozens of PMs helped surface blind spots, test unsupported headroom assumptions, identify adjacent-system effects, and improve experiences within existing scope. The team’s lessons emphasize decision frameworks, broader context, hard boundaries, prioritization, and preserving human judgment for final approval.
The Convergence of Open Table Formats and Open Catalogs: Catalog Commits is Generally Available
Catalog Commits is generally available for Unity Catalog managed Delta tables, aligning Delta with Iceberg’s catalog-oriented model and making the catalog responsible for table discovery, access, and latest table state. The change addresses three coordination problems: metadata “split brain” when engines write directly to storage, fragmented multi-engine governance, and the historical inability to coordinate atomic writes across multiple tables. With Catalog Commits enabled, Unity Catalog brokers table access through standardized APIs, keeping catalog and table state synchronized and enabling consistent authorization, holistic auditability, automated optimizations, and multi-statement, multi-table ACID transactions on Databricks. The release supports Databricks products and engines including Delta Spark, Delta Flink, Starburst Trino, DuckDB, and StreamNative, while Delta Kernel provides a shared path for connector support.
Announcing Databricks student fellows
Databricks is launching the Student Fellows Program, an opportunity for university and college students interested in computer science, AI, and data engineering to become leaders in their campus communities. Fellows receive specialized training from Databricks experts, build hands-on skills with the Databricks platform, and organize campus activities such as hackathons, tech talks, and study groups while connecting peers with resources. The initiative also offers opportunities for free or discounted certification exam vouchers and practical experience intended to support applications for internships with Databricks, its customers, or partners. Student Fellows’ primary mission is to foster a community of learners and serve as a bridge between Databricks and their university communities, with the first cohort currently accepting applications.
Faster Queries and New Capabilities with the Open-Source Databricks JDBC Driver
The open-source Databricks JDBC driver 3.x and later releases address connectivity needs affecting dashboards, spreadsheets, applications, and other data workflows. Compared with the legacy 2.x driver, they deliver up to 30% faster retrieval of large query results while preserving backward compatibility. Architectural changes add full Arrow-based transfer for JDK 16+, a first-class asynchronous execution API, stream-based ingestion into Databricks Volumes, and integration with the Statement Execution API. The driver also supports stored procedures, multi-statement transactions, Unity Catalog metric views, query tags, geospatial and complex data types, plus built-in client telemetry for query latency, metrics, and errors. Its open-source ownership and Databricks maintenance are presented as enabling faster fixes, new features, transparency, contributions, and tighter platform integration.
Top topics
- Architecture834
- Performance815
- LLMs659
- Machine Learning621
- Open Source563
- AI Agents553
- AI469
- Developer Experience448
Weekly digest
The best engineering reads, once a week.