ClickhouseProblem & solution
What else runs on your Postgres server, and how do we stop it from taking the database down?
Summary
The Postgres VM described here runs PgBouncer instances, a backup agent, metrics exporters, a local Prometheus, a log collector, and watchdog timers alongside the database. Those processes share machine-wide resources outside Postgres’s huge-page-backed shared_buffers, so leaks, CPU saturation, or disk exhaustion can threaten database availability. ClickHouse Managed Postgres places supporting services in a cgroup v2 slice and combines Go runtime GOMEMLIMIT targets with systemd memory.high and memory.max enforcement. GOMEMLIMIT encourages earlier garbage collection, memory.high applies reclaim and throttling, and memory.max provides a hard ceiling whose OOM scope excludes Postgres because it uses a separate cgroup. CPU weights, bounded buffers, log and metric controls, disk thresholds, and exemptions for replication and monitoring sessions extend isolation across other failure modes; these controls are enabled by default.
Context
The Postgres VM runs multiple supporting processes alongside the database, including connection poolers, backup and monitoring agents, exporters, logging, and watchdogs. These processes share machine-wide memory, CPU, and disk resources with Postgres, so a memory leak, CPU-intensive backup, unbounded metrics cardinality, log burst, or full disk can cause a database availability incident.
Approach / What changed
Supporting services are placed in a cgroup v2 slice with memory.high and memory.max limits, while Go services receive GOMEMLIMIT targets. Additional controls include lower CPU weight for backups, fixed WAL-G buffers, a log collector memory limiter, an allowlist for exported metrics, disk thresholds, and a watchdog that terminates ordinary application sessions at the emergency disk threshold while exempting replication and monitoring users. Postgres runs in a separate cgroup from supporting services.
Takeaways
GOMEMLIMIT gives Go services a runtime-level memory target, while memory.high triggers reclaim and throttling and memory.max supplies kernel-enforced cgroup OOM isolation.
Backup CPU weight is reduced and WAL-G buffers are fixed as a fraction of RAM, limiting backup pressure on foreground database work and bounding memory as workload grows.
At the disk emergency threshold, the watchdog terminates ordinary application sessions but preserves replication and monitoring sessions so WAL streaming and operational visibility remain available.
Source and freshness
- Source
- clickhouse.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.