Fly.ioExplainer
I'm All-In on Server-Side SQLite
Summary
Ben Johnson argues that SQLite should be considered as the core data and persistence layer for many production full-stack applications, rather than being relegated to unit tests in an n-tier architecture. He attributes this case to cheaper, faster hardware and the fact that most successful applications will never approach terabyte-scale data, making distributed database infrastructure excessive for some workloads. Litestream works alongside standard SQLite by taking control of WAL-mode journaling, capturing WAL updates, replicating them to storage such as S3, and handling checkpoints itself. This preserves ordinary SQLite libraries and transactions while providing a path to restore copies and improve resilience to storage failure; upcoming direct database-to-database replication is also mentioned. The resulting model places data in the application process, where the post reports 10–20 microsecond per-query latency, but it remains aimed at single-node applications and has restore, deployment, and replication-guarantee limitations.
Context
The conventional full-stack architecture separates an application server from a database server, while SQLite is commonly treated as suitable mainly for unit tests. The post argues that faster, cheaper hardware and the fact that most applications do not approach terabyte-scale datasets make this conventional arrangement unnecessarily heavyweight for some production workloads. It also identifies storage failure and concurrency at scale as reasons applications do not default to SQLite.
Approach / What changed
Litestream runs alongside an application using standard SQLite. It takes control of SQLite's WAL-mode journaling, prevents automatic checkpoints with an indefinite read transaction, captures and replicates WAL updates, and performs checkpointing itself. Databases can be replicated to destinations including S3 and restored elsewhere; an upcoming release is described as adding live replication directly between databases.
Takeaways
SQLite runs inside the application process, with the post reporting per-query latency of 10–20 microseconds— described as a 50–100x improvement over an intra-region Postgres query.
Litestream does not parse queries, proxy transactions, or require a new application library; it uses SQLite's existing journaling and concurrency features through a separate process.
The design targets single-node applications and is unsuitable for ephemeral serverless platforms or rolling deployments; sequential restores can take minutes, and the separate process limits replication guarantees to coarse-grained control.
Source and freshness
- Source
- fly.io
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.