How Apache DataFusion Runs SQL In-Process
Apache DataFusion is an embeddable query engine: SQL goes in, runs as columnar Arrow batches inside your own process, and results come out without a separate service.
Loading visual explainer…
Bazel in the Inner Loop: From Monorepo Graph to Focused Sync
See how Bazel’s dependency graph can separate build scope from IDE scope, helping engineers move through large monorepos without syncing every module.
Explanation
Bazel gives large monorepos a dependency-graph view of the codebase, but fast engineering work also depends on how tools consume that graph. This explainer follows the path from a code change to build analysis, metadata or language-server indexing, and a focused IDE sync. Grab’s Android workflow shows why narrowing sync scope matters, while Metals v2 shows how build-free indexing and metadata-first BSP integration can keep code intelligence off the startup path.
FAQ
Why separate Bazel builds from IDE synchronization?
A build system and an IDE need different information at different times. Grab’s Focus plugin uses a dependency graph to avoid repeating Gradle configuration, while Metals v2 uses build-free indexing and metadata-first BSP integration to keep build synchronization off the startup path.
When is a focused sync useful, and when should scope expand?
Focused sync is useful when an engineer is working within a limited module scope in a large monorepo. Grab reports that focused syncs reduced typical Android Studio sync time and memory use, but broader IDE operations can require expanding the focus or performing a full sync.
What is the role of an indexed or cached view of the repository?
Metals v2 uses a build-free, content-addressed repository index plus compiler-backed Scala and Java pipelines for code intelligence. The practical idea is to reuse repository metadata for interactive analysis instead of making every editor startup wait for a full build synchronization.
The posts behind it
Related reading
Databricks · · AnnouncementOpen-sourcing Metals v2: Databricks’ Java and Scala language server for multi‑million line codebases
Databricks open-sourced Metals v2, a Scala and Java language server designed to provide low-latency code intelligence for multi-million-line Bazel monorepos, including its 26M-line repository. The project extends Metals v1 by reworking three layers: a build-free, content-addressed mbt repository index; compiler-backed Scala and Java pipelines using a Metals-provided sourcepath; and metadata-first BSP integration that keeps build synchronization off the startup path. Its Java implementation uses javac APIs and a modified Turbine header compiler, which processes close to one million Java lines per second on one thread and supports interactive analysis at nearly 100,000 lines per second in benchmarks. Metals v2 is available in Cursor, VS Code, and Neovim under Apache 2.0, while Databricks reports Cursor’s share of Scala and Java file-open events rising from 40% to 78%.
Grab · · Problem & solutionScaling developer experience: How we improved Android Studio in a large monorepo
Grab’s passenger Android monorepo contains roughly 2,000 modules and 11 million lines of code, making Gradle-based Android Studio syncs increasingly slow and memory-intensive even though Bazel handles builds. A cold full sync could exceed 35 minutes, and a survey of 55 developers found that 82% wanted an option to exclude modules from syncing. The team built a custom Focus plugin that uses a dependency graph generated during migrateToBazel, avoids repeating Gradle’s configuration phase, and integrates with Android Studio through a one-click Select → Focus → Sync flow; a Groovy include-shadow technique avoids requiring teams to reorganize settings files. Typical focused syncs fell below two minutes, while reported results show reductions from 35 minutes to under one minute depending on module size, memory usage from 10 GB to 2 GB, and improved Compose preview updates, with broader IDE operations requiring expanded focus or full sync.
More like this
More how it works explainers
How MaaS Governance Wraps a Model Endpoint on AKS
A model deployed on AKS is open by default. See how MaaS policies insert Authorino authentication and Limitador rate limits in front of vLLM, so bad keys and over-quota requests stop at the gate.
How Caddy Serves a Static Site: The Request Flow
See how Caddy handles a request for a static site, from DNS resolution through container routing to file_server, and how TLS and redirects fit into the flow.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.