Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,668 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Encoding Your Domain Expert: The Context Layer Behind Spotify's Data Assistant
Spotify developed an AI data assistant, Vedder, to scale access to over 70,000 internal datasets without overwhelming domain experts. Because raw warehouse schemas omit critical semantic nuances and overwhelm LLM context windows, engineers implemented a domain-owned context layer organized into clusters. Each cluster contains profiled datasets, expert-curated business documentation, and vetted question-and-SQL pairs that guide a ReAct-based agent. Automated query history extraction proved insufficient on its own, as domain curators accepted only 12.5% of inferred query pairs due to noise. Deployed across Slack, IDEs, and a dedicated web interface, the system maintains reliability through continuous cluster health monitoring that flags schema drift and degraded examples.
Announcing the Databricks storage ecosystem: Governing the enterprise data estate, wherever it lives
Databricks announces a Software-Defined Storage (SDS) Ecosystem for governing enterprise data across on-premises, private-cloud, and edge environments without requiring migration. It responds to sovereignty and regulatory constraints, data-gravity economics, latency requirements, and the need to unlock AI value in backup, archive, and other previously inaccessible data. Storage partners implement the open-source OpenSharing protocol, expose an endpoint, and connect it to Unity Catalog so Databricks Serverless Compute can access governed data where it resides. The integrations provide a unified catalog and support Serverless Compute, Genie, AgentBricks, and model training with zero data movement or duplication; MinIO is generally available, while Everpure, Qumulo, and VAST Data are in preview stages. Databricks also says commitments are in place from six additional providers and that Volumes APIs are being developed to extend OpenSharing to unstructured files for generative-AI workloads.
Bugbot is now over 3x faster, 22% cheaper, and finds 10% more bugs
Bugbot has been updated to run over three times faster, cost 22% less, and find 10% more bugs per review; 90% of runs now finish in under three minutes. The /review command lets users run Bugbot or Security Review before pushing code, while /review-bugbot and /review-security invoke those agents directly. Reviews run through /review sync with Bugbot on GitHub and GitLab, allowing a matching pull-request diff to skip a duplicate review, and Bugbot can now be configured to inspect only changes since its last review. The gains come from harness improvements and training Composer 2.5, which powers Bugbot; organizations opting out of that model use the next best available model, with speed and performance varying by configuration.
Modern BSA/AML compliance on Databricks
AML operations are strained by fragmented systems, high false-positive volumes, manual case documentation, and opaque vendor scoring, leaving analysts focused on backlog rather than financial-crime intelligence. The proposed Databricks Data + AI Platform unifies transaction monitoring, KYC, sanctions, case history, and policy data under Unity Catalog, using Lakeflow Connect and a Bronze–Silver–Gold Delta architecture with masking, row-level security, and lineage. MLflow, Model Serving, Lakehouse Monitoring, and inference tables support institution-specific detection models, while Agent Bricks coordinates agents for evidence gathering, recommendations, and SAR drafting with analysts retaining final decisions. The architecture also uses Lakebase for governed operational state and Databricks Apps for analyst and executive experiences. Reported outcomes include a 75% reduction in false positives reaching the analyst queue and compressing three-to-six-hour investigations to minutes.
DiffusionGemma: The Developer Guide
DiffusionGemma is an experimental text-generation model and developer guide covering serving, customization, and non-autoregressive generation. Built on the Gemma 4 backbone, its 26B Mixture of Experts architecture activates 3.8B parameters during inference, refines 256-token canvases in parallel, and shifts the bottleneck from memory bandwidth to compute. Block autoregressive diffusion commits denoised 256-token blocks to the KV cache, combining bidirectional denoising with causal prefill for longer sequences; vLLM integration provides an OpenAI-compatible local server. A Sudoku fine-tuning example using Hackable Diffusion reports improvement from roughly 0% base-model success to 80%, while SFT reduces the demonstrated solve from 48 to 12 steps through adaptive stopping. The guide also identifies bidirectional attention and re-noising as mechanisms for global constraint handling and self-correction.
Quick: An internal hosting platform for the AI era
Shopify built Quick to make internally created prototypes, dashboards, tools, and games easier to share than deploy, giving employees a secure URL for a folder of HTML and assets. Launched internally in July 2025, it places sites behind Identity-Aware Proxy, serves bucket content through NGINX and gcsfuse, and deploys with a wrapper around gcloud rsync. A service in front of CloudSQL provides database and realtime APIs, while server-held credentials support calls to Shopify’s AI proxy; file uploads, BigQuery access, WebSockets, and employee identity are also available. Quick now hosts more than 50,000 sites, has been used by over half of Shopify employees, and runs on one $200-per-month VM, with constraints, rate limiting, and a migration from Node to Go supporting its operation.
Claude Fable 5 is now available on Databricks, fully governed through Unity AI Gateway
Claude Fable 5 is now generally available on Databricks, with rollout across AWS, Azure, and Google Cloud through Unity AI Gateway. The Mythos-class model targets long-running, complex, and ambiguous work, including autonomous enterprise workflows, document question answering, code investigation, and multimodal tasks. In Databricks' OfficeQA Pro benchmark, Fable 5 achieved 57.9% correctness, setting a state of the art; compared with Claude Opus 4.8, it was 20% more accurate and used 12% fewer tool calls, but ran approximately 30% slower and generated 2.5x more output tokens. Unity AI Gateway provides unified API access, fine-grained permissions, Unity Catalog logging, request and tool-call guardrails, and spend controls. Agent Bricks supports domain-specific agents, while Anthropic's policy includes 30-day retention for trust and safety purposes only.
From Chaos to Clarity: How We Built a Unified, Self-Routing Support Ops Ticketing System at Lyft
Lyft Urban Solutions' Support Ops team faced fragmented ticketing across four Jira Help Centers with duplicate intake forms, lack of routing logic, and no dashboards. Over five years, the team redesigned the intake architecture by replacing redundant forms with dynamic Jira Proforma forms featuring conditional branching. Automated Jira rules were introduced to assign tickets, apply a taxonomy of over 30 labels, and trigger PagerDuty alerts for P0 incidents. To unify operations across disparate backend projects, the system automatically clones and links tickets from a single portal to downstream project boards. Jira Structures and a Jira-to-Mode ETL pipeline provided cross-project rollups and business analytics before an upcoming Jira Cloud migration required dashboard rebuilds.
Announcing the winners of the 2026 Databricks Customer Awards
The 2026 Databricks Customer Awards recognize organizations and leaders using the Databricks Data + AI Platform across eight categories and four regions. The announcement names winners including Applied Materials, Virgin Atlantic, Fonterra Co-operative Group, Telefónica | Vivo, Virtue Foundation, Octopus Energy, Axpo, Atlassian, Wassym Bensaid at Rivian and Volkswagen Group Technologies, and Kenan Colson at Lippert. Examples include Applied Materials’ move from a Hadoop-based data lake to a governed lakehouse, with 1,500-plus analysts, more than 100 production machine-learning models and faster pipeline development, while Fonterra centralized data to improve supply-chain and compliance work. Rivian unified vehicle, factory and enterprise data, consolidated legacy systems into Delta and Unity Catalog, and designed for 500–600 petabytes; Lippert deployed AI tools across customer care, finance and HR. The announcement presents data and AI adoption as a cross-functional operating model.
Announcing the 2026 Databricks Customer Awards Industry winners
Databricks announced its 2026 Customer Awards Industry winners, recognizing 10 organizations across financial services, communications, health and life sciences, manufacturing, retail and CPG, energy and utilities, enterprise technology, public sector, digital-native businesses and cybersecurity. The cited work uses data and AI to address industry-specific needs, including SMBC Group’s governed lakehouse for risk and finance, Hospital for Special Surgery’s full-system ingestion strategy, Lumen’s conversational service-operations workflows and Superhuman’s high-volume model serving. Reported results include HSS ingesting more than 40 source systems and creating over 14,500 production tables, Lumen recording 3 million-plus AI-powered diagnostics and 35% ticket deflection, and Superhuman handling peaks above 200,000 queries per second. Adobe is also recognized for applying software engineering practices to cybersecurity detection workflows, reducing false positives and improving development speed.
Under the Hood: Scaling Responsible AI at Uber
Uber describes a company-wide Responsible AI program designed to govern models developed across its many platforms and teams as AI becomes central to products and services. Its five pillars are visibility, explainability, governance, education, and adoption, implemented through a centralized Model Catalog, automated metadata, feature attribution, lifecycle controls, and employee learning resources. Model Cards provide searchable model descriptions, metrics, deployment details, and documentation, while Michelangelo integrates global, local, and deep-learning explanations using PFI, SHAP-based methods, TreeSHAP, and integrated gradients. Governance is shifted left into product and engineering reviews, with completion prompts before deployment, and adoption work brings existing models into scope through lifecycle-aware, technology-agnostic, context-sensitive controls. The program gives engineers and oversight teams shared visibility into model behavior and risk while aiming to support continued AI growth without losing accountability and trust.
Scaling beyond one: How Airbnb evolved its data architecture for a multi-product world
To support its expansion from Homes into Experiences and Services, Airbnb evolved its offline data warehouse architecture to balance domain-specific needs with organizational consistency. Rather than enforcing a single global pattern, data teams adopted a framework of foundational principles that paired separate data models for unique product features with monolithic models for cross-cutting domains like payments and messaging.
Extending Fin as the most open Agent platform
Fin is being made available as a Service Agent on top of HubSpot and Freshworks, allowing HubSpot and Freshdesk customers to deploy it without migrating from their helpdesk. The announcement says customers can get it integrated and live in less than an hour across voice, email, chat, social, and other channels, with a stated 76% average resolution rate. Fin is described as handling complex queries that require reading and writing to third-party systems, while remaining configurable to each business’s policies; its openness includes APIs, MCPs, a CLI, access to the Apex trained model, and public product, help-center, and developer documentation. The broader argument is that open Agent platforms let customers experiment, replace components, and avoid vendor lock-in, with competition determined by product quality.
ClickHouse Agents: Claude-powered agentic analytics, now in public beta
ClickHouse announced the public beta of ClickHouse Agents, a fully managed agentic analytics service in ClickHouse Cloud powered by Claude. It provides a no-code builder for agents grounded in live ClickHouse data, plus a managed chat experience in the Cloud console, so users need no SQL or separate hosting setup. Built on LibreChat, the service supports natural-language questions, SQL and visualizations, sandboxed Bash, Python, and JavaScript execution, shareable artifacts, skills, memories, MCP-compatible connections, and multi-agent workflows. Claude Sonnet and Haiku are pre-configured, while enterprise controls include SSO, encrypted message storage, and governed data access. ClickHouse says its internal deployment handles roughly 70% of data warehouse queries and processes more than 45 million tokens daily, confirming its view that real-time analytics is a natural context for high-volume agent workloads.
How QRT powers real-time research and risk management at petabyte scale
Qube Research & Technologies (QRT), a global quantitative investment manager headquartered in London, uses ClickHouse Cloud for a data platform that serves researchers and for near-real-time risk monitoring, management, and P&L. Founded in 2018 as a cloud-native firm after the spinout of Credit Suisse’s proprietary trading arm, QRT chose a cloud-based approach rather than inheriting decades of on-premise infrastructure. The platform supports two major systems, while ClickStack consolidates logs, metrics, and traces for observability; alerts, log queries, and metric aggregations run against a single ClickHouse instance using standard SQL. QRT says its previous database was constrained by writer and reader counts and total cluster load, whereas ClickHouse provides greater speed and scalability, and that a unified monitoring backend simplifies maintenance while remaining reliably fast.
NYC Bike Courier Regulations FAQ
New York City couriers who deliver by bike or e-bike must meet requirements covering platform identification, protective equipment, bicycle hardware, and Department of Transportation safety training. The FAQ explains how to submit a New York or New Jersey mailing address in the Uber Driver app, receive an Uber-branded vest and photo ID, request replacements, and troubleshoot shipping or address changes. It lists required equipment including a helmet, front lamp, bell, side reflectors, and working brakes, while noting that replacements are available six months after the first order and brake reimbursements require dated, itemized receipts for eligible purchases on or after April 8, 2026. Training requires a DOT account using the Uber-associated email, a quiz, and certificate sharing through a 10-digit Worker ID or the app.
How an Agent Built a 3D Paris Gallery by Chaining Two Hugging Face Spaces
A coding agent built a 3D Paris gallery by chaining two Hugging Face Spaces: an image-generation Space produced dark-background monument images, and VAST-AI/TripoSplat reconstructed each single image as a 3D Gaussian splat. It then handled integration work by correcting the splats’ Y-down orientation, auto-framing monuments, compressing .ply files to .ksplat at about one-third the size, and building and deploying a static Three.js viewer. The Spaces’ agents.md files provided schemas, call and polling templates, file-upload instructions, and authentication guidance, allowing the agent to call them without a client library. The same pipeline produced six-monument galleries for Egypt and Japan after the prompts changed. The account presents callable, documented multimedia components as a way to reduce integration work and make new multimedia applications cheaper to create.
Okta Developer Connect San Francisco 2026 Recap
Okta Developer Connect San Francisco brought more than 100 developers, architects, founders, platform engineers, and security leaders to Okta HQ for technical sessions, hands-on labs, and community conversations about securing identity in the age of AI. The keynote announced general availability of Okta for AI Agents, which provides discovery, shadow AI detection, short-lived credentials, scoped tokens, vaulted secrets, MCP server controls, lifecycle governance, SIEM audit logs, and a kill switch. Sessions examined Cross App Access, using identity assertion and token exchange across a requesting app, identity provider, and resource app, plus Auth0-based patterns including Token Vault, asynchronous authorization, and fine-grained access control. Discussions described agents outpacing governance, static credentials creating risk, and RAG pipelines requiring data-layer permissions that match user access. The event concluded that builders should treat agents as identities with ownership, scope, review, consent, and revocation designed before integrations go live.
Migrating Your GitHub CI to Hugging Face Jobs
Trackio's GitHub Actions CI was limited by generic hosted machines, maintenance-related delays, and lack of readily available GPU runners, while the project needed CPU checks and CUDA-based tests. The setup keeps GitHub Actions as the CI coordinator and uses a dispatcher Space plus the huggingface/jobs-actions bridge to launch ephemeral GitHub runners inside Hugging Face Jobs, selected through labels such as hf-jobs-cpu-upgrade and hf-jobs-t4-small. The guide covers duplicating and configuring the dispatcher, creating a GitHub App, supplying runner credentials and an HF token, changing runs-on, choosing Docker images, and validating runs and logs; Playwright and NVIDIA CUDA images are used for CPU and GPU examples. In Trackio, the CPU job took 1m10s versus a 1m40s GitHub ubuntu-latest baseline, while the GPU check passed in 45s and cost less than a cent at the t4-small rate; logs were available through both systems.
What's new in clickhousectl v0.3.0
clickhousectl v0.3.0 adds improvements aimed at agentic workflows, additional installation methods, OAuth for Cloud service queries, and custom configuration files for local servers. It detects coding-agent environments and defaults to JSON on stdout, uses gh-style exit codes, reads CLICKHOUSE_ credentials from a working-directory .env file, and hides deprecated API fields from CLI output. Cloud service queries can authenticate with OAuth without provisioning a Query Endpoint; idled services wake automatically, while stopped services fail with a start hint. For local development, selected versions resolve from installed binaries before network lookup, local use can put ClickHouse on PATH, and named XML, YAML, or YML files overlay built-in defaults. The release also adds Python installation options and extends local init with a postgres/ tree alongside clickhouse/.
Top topics
- Architecture834
- Performance815
- LLMs659
- Machine Learning621
- Open Source563
- AI Agents553
- AI469
- Developer Experience448
Weekly digest
The best engineering reads, once a week.