Litestream Writable VFS

Summary

Litestream’s VFS is being extended to support Sprites, whose storage stack needs SQLite-backed block maps to serve writes immediately during cold starts, before a full database or block map is available. The VFS indexes each database page by file, offset, and size in LTX files, serves reads from object storage, and now optionally buffers writes locally before syncing them roughly every second or on clean shutdown. Write mode assumes a single writer, disables polling, and offers only the same eventual durability already provided by Sprite storage. A separate hydration mode serves remote reads while reconstructing a complete database file in the background, then switches reads to that file; the temporary file is discarded on exit. These features target Sprite-like workloads, while ordinary Litestream sidecar deployments do not require them.

Takeaways

1. 1

   Writable VFS mode assumes a single writer, disables polling for remote LTX changes, and does not treat writes as durable until the write buffer syncs.

2. 2

   The VFS stores page locations as file, offset, and size entries in LTX files, allowing requested pages to be fetched from object storage and cached without downloading the whole database.

3. 3

   Hydration uses LTX compaction to write only the latest page versions into a temporary full database file; reads continue during hydration, and the file is discarded when the VFS exits.