Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,668 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Uber Reveals 2025 “Airport of the Year” Award Winners
Uber’s 2025 Airport Awards recognize airports in San Antonio, San Diego, Dallas, and Pittsburgh for improving rideshare reliability, infrastructure, and rider experience. San Antonio International Airport won Quickest Pickup, reporting an average on-demand pickup time of just over four minutes after implementing a nearby staging lot, advanced dispatch technology including ExpressMatch, defined pickup zones, and efficient loading practices. San Diego International Airport and Dallas Love Field received Experience Obsessed recognition for redesigned pickup arrangements, while San Diego added a pedestrian-island setup and secondary staging lot and Dallas replaced a difficult garage location with a terminal-adjacent area. Pittsburgh International Airport won Smartest Space for pull-through loading bays, coordinated signage, Rematch and Prematch technology, and a larger driver waiting lot designed to improve matching, reliability, and pickup speed.
Practical Data Lakehouse Examples and Use Cases
The article presents practical data lakehouse patterns for streaming analytics, IoT pipelines, machine learning workflows, and enterprise reporting, addressing the gap between theoretical definitions and deployable examples. It explains how a lakehouse combines low-cost, schema-flexible object storage with schema enforcement, ACID transactions, data versioning, lineage tracking, and query performance, allowing SQL and ML workloads to use shared open-format tables. Examples include second-level fraud detection, medallion-based historical analytics, predictive maintenance from sensor data, and governed customer 360 profiles. The implementation guidance covers raw storage and partitioning, centralized catalogs, decoupled compute, role-based access, time travel, migration coexistence, SLAs, observability, and lifecycle policies. Together, these patterns are presented as a unified architecture that reduces duplication and data movement while supporting governed analytics at scale.
Data Governance Architecture: A Complete Blueprint for Modern Organizations
Data governance architecture is presented as a blueprint for aligning policies, roles, processes, and technologies with business outcomes. It defines objectives including consistent data definitions, data integrity, layered access controls, and secure self-service analytics, while assigning responsibilities across executives, architects, engineers, analysts, managers, and compliance teams. Core principles are accountability, transparency, consistency, and stewardship, supported by federated ownership through councils, data owners, and embedded stewards. The discussion compares DAMA-DMBOK, TOGAF, and Zachman according to organizational scale, regulatory context, and architecture maturity, and describes modern patterns including lakehouse, data mesh, and data fabric. It concludes that effective programs require executive sponsorship, documented roles, measurable quality controls, iterative implementation, and sustained change management.
Query Tags: The Context Your Warehouse Queries Have Been Missing
Databricks SQL Query Tags address a visibility gap in warehouse workloads: built-in logs identify users, warehouses, and tools, but not dashboards, projects, teams, or cost centers. Query Tags, now in Public Preview, attach multiple custom key-value pairs to each SQL execution, carry them into the Query History System Table, and expose them in the Query Profile UI. Partner integrations can automatically tag dbt models, while Power BI and Tableau support connection-level tags, and APIs and connectors support connection- or statement-level metadata. Users can also issue SET QUERY_TAGS in SQL Editor, notebooks, dashboards, or alerts to label subsequent session statements. These tags enable SQL or Genie queries for cost allocation, regression analysis, workload filtering, and environment comparisons; future plans include broader connector and workload support plus Query History search.
Holo3.1: Fast & Local Computer Use Agents
Holo3.1 is a family of computer-use models released to improve robustness across web, desktop, and mobile environments, agent frameworks, and deployment targets. Based on Qwen, it adds native function-calling alongside structured JSON outputs, expands mobile automation, and offers 0.8B, 4B, 9B, and 35B-A3B sizes for different cost, latency, and performance needs. On AndroidWorld, the 35B-A3B model rises from 67% to 79.3%, while the 4B and 9B variants rise from 58% to 72%; function-calling and native execution achieve near-parity across the stated benchmarks, and Holo3.1 improves more than 25% over Holo3 in the Holotab harness. The release also provides 35B-A3B FP8, Q4 GGUF, and NVFP4 checkpoints for local inference: NVFP4 matches FP8 on OSWorld at about two points below BF16, reaches 1.74 times BF16 token throughput on DGX Spark, and with harness optimizations cuts average step time from 6.8 seconds to 3.3 seconds.
Expanding Project Glasswing
Project Glasswing is expanding its collaboration from roughly 50 initial partners to approximately 150 new organizations across more than 15 countries. These partners must meet security requirements, and many operate critical infrastructure or maintain software used widely, with a major attack estimated to affect more than 100 million people for most. The program follows partners’ use of Claude Mythos Preview to scan codebases, finding more than 10,000 high- or critical-severity flaws, while some now use it to write patches and run pre-release checks. Its broader purpose is to help defenders adapt to powerful cyber models by providing tools and infrastructure, then shifting support toward disclosure, fixing, and deployment of patched software. General access will require safeguards against misuse that have yet to be developed.
Practical Data Warehouse Design and Architecture Guide
The guide presents data warehouse design as a business-aligned process for building, modernizing, or scaling analytics infrastructure, rather than merely storing data. It recommends defining analytics use cases and stakeholder needs first, then organizing a three-tier architecture of source, storage, and semantic output layers; cloud designs can decouple compute and storage and use open formats. A Bronze–Silver–Gold medallion flow preserves raw lineage, applies cleansing and deduplication, and produces consumption-ready dimensional models, while retention and archival policies control sprawl. For modeling, it favors star schemas for user-facing BI, uses snowflake normalization when redundancy is material, and stresses explicit fact-table granularity, domain-owned data marts, and workload-specific refresh cadences. Governance and operations include Unity Catalog, access controls, masking, lineage, multi-region deployment, disaster recovery, and CI/CD, followed by phased rollout through high-value domains.
AI Governance Maturity Model: Matrix, Assessment, and Roadmap
The AI governance maturity model assesses how deeply governance practices are embedded across an organization’s data, process, and people dimensions, using five stages from Ad Hoc to Optimized. It frames the model as a diagnostic and roadmap for boards and executive sponsors, while a five-dimension matrix separately scores strategy and leadership, policy and ethics, risk management, data governance, and monitoring and observability. The progression moves from discovery and basic ownership through standardized controls, quantified risk, real-time indicators, lineage tracking, and automated, context-aware enforcement. The recommended roadmap starts with a baseline within 30 days, targets Level 3 across the five dimensions within 12 months, runs a 90-day pilot on two or three high-priority systems, then scales effective controls through CI/CD integration and monitoring, with quarterly reviews and annual reassessment.
What we’ve learned building cloud agents
Cloud agents have evolved from local-agent extensions into systems running on dedicated virtual machines, with isolated environments, dependencies, network access, parallel execution, and unattended long-running work. That shift makes the development environment a product: reliable execution increasingly depends on rebuilding tools, VM hibernation and resumption, image checkpointing and forking, integrations, and controlled access to secrets and networks. After a fragile work-stealing design produced roughly one 9 of reliability, the team migrated the agent loop to Temporal, whose retries and scheduling across failures helped exceed two 9s; Temporal now handles more than 50 million actions daily across over 7 million workflows. Decoupling agent-loop, machine, and conversation state enables independent pod lifecycles, asynchronous subagents, and retry-aware streaming, while shorter workflows simplify upgrades. The next goal is self-healing environments that can report and resolve missing secrets, blocked network access, or other obstacles.
RAG vs Fine Tuning: Enterprise Decisions for AI Models and AI Systems
The guide frames enterprise choice between retrieval-augmented generation (RAG) and fine tuning as a trade-off between injecting knowledge at inference time and changing model weights before deployment. RAG connects an LLM to external stores by embedding a query, searching vector databases for similar chunks, and adding retrieved context to the prompt, making current information and citations possible. Fine tuning uses curated input-output examples and supervised training to update parameters; full-model training is expensive, while LoRA trains a smaller set of added weights and can support a single-GPU pilot. The decision depends on whether the priority is stable behavior and output format or fresh, traceable knowledge, alongside data quality, latency, compute, and maintenance requirements. For many mature deployments, the guide recommends combining both: fine tuning for domain understanding and consistency, and RAG for real-time data.
Introducing Cross-Engine ABAC
Cross-engine ABAC is announced in Beta, extending Unity Catalog's fine-grained governance to external engines through Iceberg REST Catalog APIs. It supports tag-based row filters and column masks, including conditional logic and SQL UDFs, while allowing policies to be defined once and enforced across engines. For an external query, the engine sends a scan request, Unity Catalog evaluates entitlements and applicable policies, and returns a filtered scan plan before the engine processes authorized files. Enforcement remains at the catalog layer, so engines need not implement governance logic and can use the open scan APIs. Apache Spark is supported today through Iceberg-Spark and Delta-Spark connectors, with Starburst and DuckDB integrations coming soon; the Beta also points toward Apache Iceberg label exchange for future governance metadata sharing.
Welcoming Ben Anderson, our new Ops Generalist
Resend announces Ben Anderson as its new Ops Generalist, based in London, UK, after he built operating infrastructure at startups, most recently at Treecard. He will develop vendor, finance, and compliance operations as the company scales, bringing experience from a discipline he describes as cross-functional, hands-on, and centered on ownership across the business. Anderson says Resend’s team, product and culture, and transparent values, philosophy, and decision-making process influenced his decision to join. He also discusses running and walking as ways to regain clarity, names 1Password as his favorite tool, advises ambitious operators to seek ownership, prioritise ruthlessly, and learn from experienced operators and mentors, and values teaching complicated topics clearly.
Building a .NET API Gateway with ClickHouse and Aspire
The tutorial builds a .NET distributed application around a YARP API gateway, two backend APIs, an analytics API, a Blazor dashboard, a load generator, and ClickHouse. Aspire’s AppHost coordinates these resources, injects connection strings and service endpoints, orders startup with dependency health checks, and exposes selected services through stable or external endpoints. The gateway records proxied requests in ClickHouse while emitting OpenTelemetry metrics for request counts, duration, and backend failures. A bounded queue and background ingestion keep request logging off the hot path, while a narrow schema and materialized view precompute minute-level aggregates for volume, p95 latency, error rates, and route comparisons. Aspire supplies live telemetry, whereas ClickHouse retains request history for analytical queries and drill-down into slow requests.
TPC-H for less than a cent: ClickHouse Cloud vs. Snowflake, Databricks, BigQuery, and Redshift
ClickHouse Cloud is benchmarked against Snowflake, Databricks, BigQuery, and Redshift on the join-heavy TPC-H workload, with the main comparison using SF100: 100 GB, 866M rows, and 22 queries. Using a single 59-core AWS compute node, the benchmark measures fastest hot runs from three executions with result caching disabled, applies public billing models and comparable Enterprise-tier regional pricing, and publishes scripts, queries, and result files for reproduction. ClickHouse completed SF100 in 19.8 seconds for $0.063 and ranked first on the cost-performance score, while Snowflake Large was faster at 15.9 seconds but cost $0.143; at SF10, ClickHouse ran the workload in 2.9 seconds for $0.009. The results are attributed to two years of join engineering, including faster hash joins, improved planning, correlated subquery support, lazy column replication, runtime filters, and statistics-based join reordering.
Personalizing Genie Code with instructions, skills, memory, and MCP
Genie Code adds personalization through persistent instructions, task-specific Agent Skills, shared repository conventions, and MCP servers. Instructions apply across every agent-mode session, while AGENTS.md and CLAUDE.md files can be discovered automatically for team conventions. Skills are markdown-based packages with names, descriptions, guidance, reusable code, or scripts, scoped to workflows and available personally or through Workspace/.assistant/skills/. MCP servers expose external tools, data, and workflows from managed services, Unity Catalog-connected servers, or Databricks Apps, with workspace admin controls and permission-based user selection. Managed OAuth flows for Google Drive, SharePoint, and GitHub are currently in beta, while the MCP marketplace and Genie Code settings support server discovery and setup.
ClickHouse Release 26.5
ClickHouse 26.5 is a monthly release containing 38 new features, 51 performance optimizations, and 224 bug fixes. It reports a record number of performance optimizations and introduces a filesystem table function for running SQL against a local file system. Its main query-planning examples push ORDER BY … LIMIT through eligible LEFT or RIGHT JOINs when the sort key depends only on the preserved side, allowing ClickHouse to limit rows before joining. In a TPC-H benchmark at scale factor 100, enabling this optimization reduced fastest runtime from 1.878 seconds to 0.092 seconds, peak memory from 1.88 GiB to 10.98 MiB, and data read by 1.5×. Additional changes include a GROUP BY … LIMIT shortcut for eligible unordered queries, an experimental in-browser clickhouse-client, and per-subquery query-cache controls.
Anthropic confidentially submits draft S-1 to the SEC
Anthropic, PBC says it has confidentially submitted a draft registration statement on Form S-1 to the U.S. Securities and Exchange Commission for a proposed initial public offering of its common stock. The filing preserves an option to go public after the SEC completes its review, but the offering remains dependent on market conditions and other factors. Anthropic has not set the number of shares or the offering price. Published under Rule 135 of the Securities Act of 1933, the announcement states that it is not an offer to sell securities or a solicitation to buy them; any offers, solicitations, or sales would occur only under the Act’s registration requirements.
Introducing Mellum2: A 12B Mixture-of-Experts Model by JetBrains
JetBrains is releasing Mellum2, a 12B-parameter Mixture-of-Experts model trained from scratch on natural language and code. It targets latency-sensitive, high-throughput workloads including routing, retrieval-augmented generation (RAG), summarization, sub-agents, and coding features. The model has 12B total parameters but activates 2.5B per token, with its architecture keeping total capacity high while making inference more efficient for real-time use. Released under the Apache 2.0 license, Mellum2 is described as competitive with similarly sized open models on code generation, reasoning, science, and math benchmarks while delivering more than 2x faster inference. Its text-and-code focus supports high-throughput production workloads and private deployments where proprietary code or internal data may be involved.
Anthropic raises $65B in Series H funding at $965B post-money valuation
Anthropic says it has raised $65 billion in Series H funding at a $965 billion post-money valuation, with the round led by Altimeter Capital, Dragoneer, Greenoaks, and Sequoia Capital. The company says its run-rate revenue crossed $47 billion earlier this month and that the financing will support safety and interpretability research, additional compute, and expanded products and partnerships. It reports agreements for up to five gigawatts of new Amazon capacity, five gigawatts of next-generation TPU capacity with Google and Broadcom, and GPU access in SpaceX’s Colossus 1 and Colossus 2; AWS remains its primary cloud provider and training partner. The round includes $15 billion in previously committed hyperscaler investments, including $5 billion from Amazon, alongside strategic infrastructure partners Micron, Samsung, and SK hynix.
Debunking 8 data layout myths: why Liquid Clustering outperforms partitioning
Partitioning has long been the standard physical layout for Hadoop- and Hive-era processing, but the post argues that changing Lakehouse workloads make fixed choices brittle. Liquid Clustering treats keys as guidance for file organization, allowing layouts to evolve, supporting low- and high-cardinality filters, maintaining file sizes, and enabling row-level concurrency. Using transaction-log metadata and per-column statistics, the post says modern Delta and Iceberg engines prune files rather than directories, and reports 35% lower clustering time, 22% faster queries, roughly 90% faster metadata-only deletes, and 23-minute planning for a 10-PB table. Conversion examples include Bolt’s 138% higher write throughput and up to 63% faster reads, while a 1.1-PB workload saw 5.9x faster queries, 86% fewer bytes read, and a 27% smaller table after clustering by date, hour, source, and id.
Top topics
- Architecture834
- Performance815
- LLMs659
- Machine Learning621
- Open Source563
- AI Agents553
- AI469
- Developer Experience448
Weekly digest
The best engineering reads, once a week.