How Apache DataFusion Runs SQL In-Process
Apache DataFusion is an embeddable query engine: SQL goes in, runs as columnar Arrow batches inside your own process, and results come out without a separate service.
Loading visual explainer…
How a Board of Directors Works: The Governance Control Loop
Boards of directors turn ownership into a governance control loop: shareholders elect directors, directors set strategy and guardrails, committees verify independently, and management executes inside those limits.
Explanation
A board of directors is the layer between owners and operators. Shareholders elect directors, who set strategy and risk guardrails, delegate bounded work to committees such as audit, and hold management accountable through reporting and independent verification. The pattern matters to engineers because governed platforms use the same shape: centrally published policy, bounded delegated execution, and tracing or audit that closes the loop. Understanding the loop makes it easier to read why companies upgrade governance before scaling.
FAQ
What does appointing an independent director actually change?
It adds verification capacity outside management. ClickHouse's new independent director also chairs the Audit Committee, and the appointment came alongside a CFO hire, together signalling a deliberate build-out of finance and governance reporting as the company scales.
How is a board like an engineering governance layer?
Both set policy once and delegate bounded execution. Databricks' Unity Gateway CLI lets admins publish approved agents, models, and spend policies that developers consume with one command, with tracing showing what actually ran. A board works the same way: a written charter plus an audit trail.
When is it worth adding board-level governance?
Usually when scale makes unmanaged decisions expensive. ClickHouse built out finance and governance after passing $350 million in annualized revenue, a $15 billion valuation, and more than 4,000 customers, adding an independent director who had guided three companies through initial public offerings.
The posts behind it
Related reading
Databricks · · AnnouncementDeploy and manage coding agents at scale with the Unity Gateway CLI
Organizations face a challenge as new frontier models appear roughly every five days, making it difficult to balance flexibility with governance. Databricks introduces the Unity Gateway CLI, which lets admins centrally configure approved coding agents, models, MCP servers, skills, and spending policies. Developers launch any approved agent with a single ug command, and the CLI handles authentication and applies the published settings. Features include Smart Routing that selects cost-appropriate models per task, budget-aware defaults that switch to lower-cost options as spending thresholds are reached, and tracing to capture tool calls and skill invocations for waste analysis. In internal use, the system processed over 61 billion input tokens across roughly 360,000 requests, and tracing helped identify bugs that saved an estimated 1.2 million per year.
Docker · · AnnouncementFrom Dockerfile to Kit: the Docker Sandboxes Kit Specification
The Docker Sandbox Kit Specification v3 defines how an agent declares the networks, credentials, and volumes it needs, so these grants travel with the artifact rather than living in shell history or dashboards. A Kit is an ordinary OCI image whose manifest carries a descriptor annotation vnd.docker.sandbox.kit.descriptor, so it builds with docker buildx, pulls with docker pull, and gets scanned and signed by existing tooling. Capability types like network-policy@2 and credential@1 are versioned independently, and each normative statement in the specification is covered by a conformance check or a written waiver. Two Kit types exist: a workload supplies the root filesystem, while a mixin adds overlays such as a CLI with its network rule; the runtime rejects required capabilities it cannot satisfy. Composition uses strict provides/requires resolution, so authority changes appear as diffs, and a conforming runtime can gate upgrades that widen the grant set.
- Databricks · · Problem & solution
How I built agent-based security reviews on Databricks
Security reviews at Databricks relied on automation but still consumed expert time on predictable work, leaving less room for genuinely novel, high-risk architectures. The author built an agent-based layer on Databricks using Unity Catalog for governed tables containing standards, request data, and decisions; Databricks-hosted foundation models (Claude Haiku, Sonnet, Opus) for classification and reasoning; and Lakeflow Jobs for orchestrating notebook-based workflows on serverless compute. A conversational intake app built with Databricks Apps turns plain-language descriptions into structured requests and provides preliminary risk indications before a formal request is created. Seven focused agents each have a bounded responsibility—intake, risk assessment, requirements, specialized reviews, validation, workflow, and learning—ensuring behavior remains inspectable and testable. Routine requests that once waited days in the queue can now be completed in minutes, and security engineers spend more time on designs requiring genuine expertise.
Meta · · AnnouncementBringing Private Processing to Meta AI Glasses
Meta AI glasses require cloud processing for advanced AI features like streaming transcription and long-term recall, but traditional cloud architectures expose data during use. Private Processing uses confidential virtual machines with Trusted Execution Environments (TEEs) on CPUs and GPUs to keep data encrypted even while being computed. Before sending context, glasses perform remote attestation, verifying the server’s TEE certificate against a public transparency ledger that records all deployed binary images. Non-targetable routing via blind-signed tokens and third-party OHTTP relays prevents tying requests to specific users, while encrypted storage ensures data is only accessible with user-provided keys. This infrastructure, already proven for WhatsApp and Meta AI, extends to glasses, with security audits by firms like NCC Group and a bug bounty program for independent research.
Clickhouse · · AnnouncementClickHouse appoints Mike Scarpelli, former Snowflake and ServiceNow CFO, to Board of Directors
ClickHouse appointed Michael Scarpelli, former CFO of Snowflake and ServiceNow, to its Board of Directors as an independent director and chair of the Audit Committee. Scarpelli previously guided three companies through initial public offerings and said ClickHouse is redefining the category for real-time and agentic workloads. The company has surpassed $350 million in annualized revenue, up from roughly $200 million at the end of 2025, and achieved a $15 billion valuation in its Series D round, with over 4,000 customers including DoorDash, Meta, Tesla, and Visa. ClickHouse now employs nearly 800 people across 27 countries, plans to reach 1,000 by year-end, and derives more than half its revenue from outside the United States as it scales its enterprise motion. The appointment is part of a deliberate build-out of finance and governance capabilities, including the 2025 hire of CFO Jimmy Sexton, who also came from Snowflake.
More like this
More how it works explainers
How MaaS Governance Wraps a Model Endpoint on AKS
A model deployed on AKS is open by default. See how MaaS policies insert Authorino authentication and Limitador rate limits in front of vLLM, so bad keys and over-quota requests stop at the gate.
How Caddy Serves a Static Site: The Request Flow
See how Caddy handles a request for a static site, from DNS resolution through container routing to file_server, and how TLS and redirects fit into the flow.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.