---
title: "Improving Lakebase Postgres compute cache"
description: "Lakebase Postgres uses disaggregated storage backed by object storage, so compute-side caching must deliver low latency and high throughput while avoiding inefficient memory use. Its original design paired a conservatively sized PostgreSQL shared buffer with a local file cache (LFC), but the roughly 1 GB shared-buffer cap pushed many hits to slower NVMe storage and could create double buffering. For fixed-size computes with CU >= 80, the new configuration disables the LFC and sets shared buffers to 75% of DRAM, using explicit 2 MB HugeTLB pages to control page-table and TLB overhead. Benchmark tests found huge pages reduced tail read latency by up to ~40% and CPU use by up to ~30%, while production examples reported doubled throughput, 5× fewer storage reads, or 5× lower CPU use. Work is underway to dynamically resize shared buffers and huge pages for autoscaling computes."
---

# Improving Lakebase Postgres compute cache

[Databricks](https://yomu.fyi/company/databricks) · David Wein, Sunil Kamath, Haoyu Huang · Sep 10, 2026

**Type:** Problem & solution

## Summary

Lakebase Postgres uses disaggregated storage backed by object storage, so compute-side caching must deliver low latency and high throughput while avoiding inefficient memory use. Its original design paired a conservatively sized PostgreSQL shared buffer with a local file cache (LFC), but the roughly 1 GB shared-buffer cap pushed many hits to slower NVMe storage and could create double buffering. For fixed-size computes with CU >= 80, the new configuration disables the LFC and sets shared buffers to 75% of DRAM, using explicit 2 MB HugeTLB pages to control page-table and TLB overhead. Benchmark tests found huge pages reduced tail read latency by up to ~40% and CPU use by up to ~30%, while production examples reported doubled throughput, 5× fewer storage reads, or 5× lower CPU use. Work is underway to dynamically resize shared buffers and huge pages for autoscaling computes.

## Context

Lakebase Postgres stores data in a disaggregated storage layer and uses compute nodes as caching layers. The prior shared buffers plus local file cache design capped shared buffers at 1 GB on fixed computes, causing larger working sets to rely on slower local file storage, while standard filesystem caching could also create double buffering. Large shared buffers introduced page-table, TLB, and process memory-management overhead, especially in the process-per-backend architecture.

## Approach / What changed

For fixed-size computes with CU >= 80, the implementation disables the local file cache and sizes PostgreSQL shared buffers to 75% of DRAM. It adds explicit 2 MB HugeTLB page backing across the host, hypervisor, and guest kernel, releasing surplus pages at startup. The team is developing a protocol to scale huge pages with dynamic shared buffers on autoscaling computes.

## Takeaways

- Fixed-size computes with CU >= 80 now use shared buffers sized to 75% of DRAM instead of the local file cache; an 80 CU endpoint should report shared\_buffers as 20971520.
- Using 2 MB huge pages reduced page-table sizes by a factor of 512; benchmark results showed up to ~40% lower tail read latency and up to ~30% lower CPU utilization.
- Production examples included doubled throughput, storage GetPage/s falling from about 8K to 1.5K, and CPU use dropping from 20 cores to 4 on one workload.

**Tags:** [Caching](https://yomu.fyi/topic/caching), [Linux](https://yomu.fyi/topic/linux), [Performance](https://yomu.fyi/topic/performance), [Postgres](https://yomu.fyi/topic/postgres)

- Source: [Databricks](https://www.databricks.com/blog/improving-lakebase-postgres-compute-cache)
- Source URL: https://www.databricks.com/blog/improving-lakebase-postgres-compute-cache
- Ingested by Yomu: 2026-09-10T20:03:16.739Z

[Read original post](https://www.databricks.com/blog/improving-lakebase-postgres-compute-cache)