Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,668 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
How do you turn AI coding chaos into a repeatable playbook?
Software development dynamics are shifting as code generation costs decrease, requiring engineering organizations to adapt their leadership and processes. Snowflake addressed this transformation across its software lifecycle by first encouraging widespread, unrestricted experimentation with coding agents before formalizing successful practices. The engineering organization codified these practices into fourteen distinct AI design patterns, including strategies such as planning in markdown and isolating parallel agents. This structured approach, combined with dedicated focus weeks and a progress scale, allowed engineering teams to accelerate both inner and outer loops of development. Consequently, a team of three engineers leveraged coding agents to deliver a 40x improvement to Snowflake's query compiler while significantly shortening release validation times.
Ranch, Refills, and Viking Rows: Inside Football Fan Culture
Consumer Research tracked close to 32.6 million tournament-related mentions from June 10 to June 28 to examine how fans experienced the tournament through food, beverages, celebrations, and public events. International visitors reacted strongly to American staples: ranch dressing reached 70.3k mentions, free refills 45k, free or unlimited bread more than 30k, and barbecue nearly 64k, while Google interest in “ranch dressing” rose 586.4% and “ranch dressing ingredients” 600%. The experience also exposed friction, with stadium beers costing more than $20 and hydration breaks drawing negative sentiment in over 79% of sentiment-categorized mentions. Watch parties generated over 226k mentions, and New York held 22% of host-city watch-party conversation, offering alternatives to stadium prices. Norway’s “Viking Row” reached nearly 229k mentions, showing how organic fan rituals can spread across borders and create fast-moving opportunities for brands.
ClickHouse Agents is now available for Managed Postgres
ClickHouse Agents is now available for all Managed Postgres services in ClickHouse Cloud, extending its plain-English agent interface to operational data. The beta service, powered by Claude and built on LibreChat, combines an agent builder and chat interface with a sandboxed code interpreter; it is read-only by default and supports MCP-compatible systems. Agents can inspect Postgres schemas, run read-only queries, return the SQL used, and reason across Postgres and ClickHouse by querying each engine independently before merging results. Additional skills guide migration from Postgres to ClickHouse through discovery, access-pattern analysis, schema design, type mapping, CDC ingestion, and validation, while other workflows use metrics, Query Insights, and EXPLAIN to rank performance fixes. Planned extensions include scheduled recommendations and safe automated remediation, but the current release focuses on exploration, analysis, monitoring, and guided database work.
Keeping funds where they belong: Implementing fund segregation in Stripe Connect
Marketplaces using Stripe Connect’s separate charges and transfers can pool seller funds with platform revenue, exposing them to payouts, unrelated disputes, fees, refunds, and operational mistakes. In response to anticipated PSD3 changes and tighter Commercial Agent Exemptions, Stripe introduced Segregated Separate Charges and Transfers, which places captured payment funds in an allocated balance that cannot fund payouts and can only be transferred to connected accounts. The integration enables allocated_funds on PaymentIntents, transfer_group tracking, source_transaction-linked transfers, and application fees collected from the same allocation; balances are inspected through the API. The post also details split transfers, refund and reversal ordering, dispute handling, prerequisites, and limitations including card-only support, preview-header requirements, and sandbox-only testing, presenting segregation as a way to make fund flows traceable and operationally safer.
How we keep GPUs reliable across Databricks AI
Databricks AI describes GPU reliability as a fleet-scale problem involving crashed jobs, silent slowdowns, and numerical corruption during distributed training. Its approach combines stress testing with demanding reinforcement learning, agentic coding, and document intelligence workloads, plus gpu-monitor, a multi-stage service that checks nodes during bootstrap, continuously while active, and periodically between workloads. The checks cover compute burn-in, peer connectivity, NCCL correctness and bandwidth, RDMA, ECC and HBM memory, PCIe integrity, throttling, thermal conditions, XID errors, and fabric health. A production incident showed why cumulative InfiniBand port downtime matters: a single outage exceeded the lower-level NCCL_IB_TIMEOUT, killing a collective before the longer PyTorch watchdog timeout. Nodes that fail checks are quarantined, while inter-node probes sweep message sizes from 8 bytes to 2 GiB and apply latency or BusBW criteria; the system is continuously expanded as new failure modes emerge.
Celebrating the Winners of the 2026 Built-On Databricks Startup Challenge
The 2026 Built-On Databricks Startup Challenge recognized early-stage startups building B2B applications on Databricks, with winners presented at the 2026 Data + AI Summit. VisionHeight won the Grand Prize for an agentic threat-intelligence platform that maps adversary infrastructure across the Internet while it is being constructed, aiming to give defenders earlier warning. Linkup took second place with a production-grade Web Search API that independently crawls and indexes the open web at the fact level, delivering sourced results in about two seconds. Intelo placed third with five coordinated AI Agent Teams covering retail merchandising and planning, while Clarecast, Gemini Sports, and LakeFusion received Honorable Mentions. Judges assessed market potential, founding-team caliber, and innovative Databricks use; the announced Startup Program also offers qualifying startups up to $200,000 in combined Databricks and Neon credits.
Securing Heroku CLI Credentials with System Keychain Storage
Heroku CLI 11.8.0 changes credential storage by making the system keychain the default and retaining .netrc as a fallback. The transition began in 11.6.0, when the CLI stored tokens in both locations to preserve existing workflows. The credential manager uses macOS security, Linux Secret Service through secret-tool, or Windows PasswordVault, storing credentials under “heroku-cli” with the account email as the identifier. Authentication lookup prioritizes HEROKU_API_KEY, then HEROKU_NETRC_WRITE, keychain storage, and .netrc files; unsupported or failing keychain access falls back to .netrc. Version 11.8.0 also adds a Git credential helper, so commands such as git push heroku main continue without configuration changes, while CI/CD can keep using HEROKU_API_KEY and Linux users may need secret-tool.
Redeploying Claude Fable 5
Access to Claude Fable 5 and Mythos 5 was restored after US export controls had temporarily suspended both models for all users. The post says Fable 5 will return globally on July 1 across Claude Platform, Claude.ai, Claude Code, and Claude Cowork, while Mythos 5 access has resumed for some US organizations and broader partner access remains under discussion. Anthropic describes a new safety classifier intended to block the bypass reported by Amazon researchers in more than 99% of cases; blocked Fable 5 requests are sent to Opus 4.8, although the classifier increases false positives in coding and debugging. It also outlines defense-in-depth safeguards, a proposed shared industry framework for assessing jailbreaks, and expanded government collaboration on pre-release evaluation, information sharing, and joint research.
Claude Fable 5 and Claude Mythos 5
Claude Fable 5 and Claude Mythos 5 are presented as a joint launch of the same underlying model, with Fable 5 intended for general use and Mythos 5 initially reserved for selected cyberdefenders and infrastructure providers. Fable 5 uses conservative safeguards that route some sensitive queries to Claude Opus 4.8, while Mythos 5 lifts safeguards in selected areas through Project Glasswing and a planned trusted-access program. The post reports strong results in software engineering, knowledge work, vision, long-context tasks, cybersecurity, protein design, molecular biology, and genomics, including a reported 50-million-line Ruby migration completed in a day. Both models are priced at $10 per million input tokens and $50 per million output tokens, with Fable 5 available through the Claude API and staged subscription access.
Granular Usage Attribution for dbt Pipelines with Query Tags
Databricks Query Tags give dbt pipelines granular usage attribution when identical query-history rows obscure model ownership, cost, and performance. The dbt-databricks adapter version 1.11+ automatically injects model, materialization, dbt-core, and adapter-version metadata, while profile-level and model-level configurations add custom dimensions such as team, cost center, project, and environment. These tags are stored as a MAP<STRING, STRING> in system.query.history, where SQL can extract them for cost attribution, performance debugging, monitoring, and dashboards. In the reference project, four mart tables accounted for 92% of compute time, whereas staging views and metric views were near-instantaneous. The accompanying open-source project demonstrates Genie exploration, a self-monitoring AI/BI dashboard, metric-view tagging, and deployment with Databricks bundles, while recommending consistent profile hierarchies, environment tags, and limited custom metadata.
Beyond dashboards: Introducing Decision Execution Platforms
Databricks Forward Deployed Engineering introduces Decision Execution Platforms (DEPs), an enterprise analytics category intended to connect KPI signals, executive decisions, operational execution, and measured outcomes. The proposal addresses workflows in which dashboards reveal problems but meetings, decks, spreadsheets, and messaging threads leave implementation fragmented and impact measurement disconnected. DEPs run the four-stage loop on governed Databricks infrastructure: agents recommend actions, alternatives, predicted impact, and reasoning; approved choices execute through systems of record; and results persist in a Decision Log for continuous learning. Their architecture combines a foundation of Lakebase, Genie, Unity Catalog, Lakehouse, Agent Bricks, and MLflow with an SDK of reusable primitives and a Databricks Apps executive surface. A retailer case used a DEP to unify fulfillment data and enable simulated, controlled rerouting, with scaling aimed at measurable bottom-line and customer-satisfaction outcomes.
Mastering Agentic Techniques: AI Agent Reinforcement Learning
Language model agents often fail across long-horizon workflows, repeat tool-call errors, and output invalid schemas when relying solely on prompting or retrieval. Reinforcement learning with verifiable rewards provides a practical training signal for domain-specific tasks by scoring model rollouts against deterministic verifiers such as schema checkers, unit tests, and execution sandboxes. Teams can implement group relative policy optimization to sample multiple trajectories per prompt and update policy weights based on relative performance within the group. Establishing environments with explicit harness, state, and verifier definitions allows agents to learn multi-step tool use, whereas static datasets fail to capture sequential dependencies. Inspecting checkpoint rollouts and testing against held-out tasks prevents reward hacking while continuously turning operational failures into verifiable benchmark environments.
How to measure the customer experience as AI scales
CSAT captures fewer than 10% of conversations, and its responses skew toward delighted or furious customers. Most customers say nothing, leaving teams to assess customer experience from an unrepresentative sample that compresses product, policy, and service issues into one score. As AI handles more conversations end-to-end, the post argues for scoring every AI and human interaction on service quality, resolution, customer effort, and related attributes rather than relying on survey responses alone. Fin’s CX Score assigns each interaction a 1–5 score and surfaces reasons behind it, providing roughly five times more coverage than CSAT for most teams. Fin sets targets from CX Score attributes, operational correlations, historical performance, modelling, and team ambition, then uses recurring patterns to route fixes to support, product, or workflow owners.
Meta’s AI Storage Blueprint at Scale
Meta describes how its BLOB-storage architecture evolved to address AI workloads' storage bottlenecks, which can stall GPUs and slow research iteration as datasets and GPU deployments expand. Legacy requests traversed multiple stateful metadata layers, sometimes across regions, producing latencies of hundreds of milliseconds even when flash made millisecond access possible. The redesign consolidates metadata into a flat schema backed by ZippyDB for O(1) path resolution, removes the dataplane proxy in favor of a fat client SDK, and deploys regional BLOB-storage stacks beside GPUs. For data loading, Meta combines host memory and flash with regional flash and global HDD-backed storage as tiered caches, using ordinary and deep prefetch, on-demand hydration, and TTL or LRU eviction policies. Production rollout saw rapid adoption, while both data-loading paradigms remain supported.
amazon.science
How Amazon tracks carbon intensity across its operations
Amazon measures carbon intensity across its diverse business operations to track progress toward its goal of reaching net-zero carbon by 2040. For retail operations, the company monitors carbon emissions per unit shipped, achieving a 39 percent reduction by the end of 2025 compared to 2019 baselines. This per-unit decrease stems from targeted investments in electric delivery vehicles, carbon-free energy, smarter package routing, lightweight packaging, low-carbon fuels, and alternative transport modes like rail. To improve industry comparability and transparency, the company updated its economic carbon intensity indicator in 2025 from emissions per dollar of gross merchandise sales to emissions per dollar of revenue. Carbon intensity is also evaluated regionally and nationally to isolate geographic variations and implement localized decarbonization interventions across supply chain networks.
How we scale PgBouncer in ClickHouse Managed Postgres
PgBouncer is single-threaded, so a single process can bottleneck connection pooling on multi-vCPU machines even when Postgres has capacity. ClickHouse Managed Postgres runs a fleet of PgBouncer processes sized to available cores, sharing one port through so_reuseport; peering forwards query-cancellation requests to the process that owns the session, while transaction pooling and per-process connection-budget splits limit Postgres connections. In tests on identical AWS EC2 c7i.4xlarge pooler instances, a 16-process fleet reached roughly 336,000 transactions per second at 256 clients, compared with 76,893 for one process, while using about 49% versus 8% in-guest CPU. The fleet's advantage emerged under concurrency, whereas the single process was slightly faster at eight clients, and the described setup ships by default with every ClickHouse Managed Postgres server.
Forecasting at the speed of modern retail
Retail and CPG forecasting now spans hundreds of thousands, sometimes far more, time series across fragmented channels, promotions, and short-lived products, making legacy methods and manual exception management difficult. Multi-model forecasting addresses this complexity by evaluating a range of techniques against actual data and selecting the best-performing model for each series, but enterprise-scale experiments require scarce forecasting and distributed-systems expertise. Released in 2024, Databricks’ open-source Many Model Forecasting (MMF) integrates more than 35 statistical, deep-learning, and foundation time-series models and runs on distributed Databricks compute. MMF Agent, built on Genie Code, guides users through data quality, series classification, compute configuration, forecasting, post-processing, and model selection, while Unity Catalog helps it use organizational data context. The workflow is intended to reduce setup from days to hours, improve targeting and accuracy, and make rigorous forecasting more accessible while remaining customizable for technical teams.
How Artemis Security runs 69x faster detection queries with ClickHouse Cloud
Artemis Security uses ClickHouse Cloud on AWS for real-time threat detection across many terabytes of compressed logs, tens of billions of daily rows, and queries over trillions of rows. To reduce repeated scans, its engineers coalesce hundreds of detection-rule predicates into one query, tagging matching rows by rule ID; batches of more than 100 rules now run in under a second, with a measured 69x speedup. A Claude Code skill called /poke connects ClickHouse system tables, CloudWatch, Datadog, application data, and source code to investigate infrastructure issues through natural-language prompts. For investigative queries over semi-structured JSON, Artemis extracts frequently queried fields into a parallel materialized table and uses a hybrid two-step lookup, producing 30–60x faster queries and up to 400x lower CPU consumption.
Celebrating the Winners of the 2026 Built-On Databricks Startup Challenge - Cloned
The 2026 Built-On Databricks Startup Challenge recognized early-stage startups building B2B applications on the Databricks platform, with winners announced at the Data + AI Summit in San Francisco on June 16. VisionHeight won the grand prize for an agentic threat-intelligence platform that maps adversary infrastructure across the Internet while it is being built; Linkup placed second with a production-grade Web Search API delivering results in about two seconds, and Intelo placed third with an agentic workforce for retail merchandising and planning. Clarecast, Gemini Sports, and LakeFusion received honorable mentions for predictive intelligence, football squad planning, and an AI-native data foundation combining MDM, PIM, and LakeGraph. Judges assessed market potential, founding-team caliber, and innovative use of Databricks, while the startup-program offer provides qualifying startups up to $200,000 in credits across Databricks and Neon.
Reading the agent traces is how you make the call your eval can't
Automated evaluations often miss unknown failure modes in AI agents because developers only write tests for anticipated errors. While building an itinerary agent for AI Engineer World's Fair, an open-weight model routed a speaker query to a tool returning only tracks, leading the model to hallucinate famous speakers and falsely claim the tool provided that data. Because a tool was executed, the trace appeared grounded until inspectable tool outputs revealed the discrepancy. To resolve this, the author retained the inexpensive model, corrected query routing, tightened system prompts against ungrounded entities, and added a custom entity-diffing eval in CI. Ultimately, directly reading agent telemetry traces alongside errors and logs is essential for diagnosing root causes and deciding trade-offs between model costs, prompt adjustments, and routing fixes.
Top topics
- Architecture834
- Performance815
- LLMs659
- Machine Learning621
- Open Source563
- AI Agents553
- AI469
- Developer Experience448
Weekly digest
The best engineering reads, once a week.