---
title: "Building fast filesystems for VMs"
description: "Modal’s VM Sandboxes originally used virtio-fs on top of ImageFS, a lazy-loading FUSE filesystem. This stack caused severe performance degradation: recursive chown took over a minute instead of seconds, and tools like bun install and ripgrep ran three times slower. Analysis revealed that every metadata operation like chown required three round trips from the guest VM to the host, each incurring approximately 99 µs overhead. Modal replaced virtio-fs with ImageFS-blk, a block-level solution that presents a virtual block device to the guest, letting the kernel manage its own filesystem. The new approach delivers dramatic speedups: chown dropped from 9.87 to 0.21 seconds, and bun install fell from 12.5 to 1.07 seconds, often exceeding even direct ImageFS performance."
---

# Building fast filesystems for VMs

[Modal](https://yomu.fyi/company/modal) · Nitin Maddi · Oct 16, 2026

**Type:** Problem & solution

## Summary

Modal’s VM Sandboxes originally used virtio-fs on top of ImageFS, a lazy-loading FUSE filesystem. This stack caused severe performance degradation: recursive chown took over a minute instead of seconds, and tools like bun install and ripgrep ran three times slower. Analysis revealed that every metadata operation like chown required three round trips from the guest VM to the host, each incurring approximately 99 µs overhead. Modal replaced virtio-fs with ImageFS-blk, a block-level solution that presents a virtual block device to the guest, letting the kernel manage its own filesystem. The new approach delivers dramatic speedups: chown dropped from 9.87 to 0.21 seconds, and bun install fell from 12.5 to 1.07 seconds, often exceeding even direct ImageFS performance.

## Context

Modal previously built ImageFS, a distributed lazy-loading FUSE filesystem for containers that enabled sub-second cold starts. When they moved to VM Sandboxes, they initially used virtio-fs to share the ImageFS-backed filesystem with the guest kernel. This added an extra hop for every filesystem operation, causing severe slowdowns—especially for metadata-heavy workloads like chown and package installs.

## Approach / What changed

ImageFS-blk replaces the virtio-fs FUSE layer with a virtual block device. The guest VM runs its own ext4 filesystem on this device, handling all metadata operations locally. On the host, ImageFS-blk serves blocks from ImageFS’s tiered cache, with written blocks stored in a sparse file. Block-level copy-on-write bounds write overhead to a single block instead of the full-file copy required by OverlayFS.

## Takeaways

- Virtual machine filesystems using virtio-fs suffer from multiple host round trips per syscall because the guest kernel must consult the host for metadata, causing orders-of-magnitude slowdowns on operations like chown and recursive copies.
- ImageFS-blk replaces the FUSE-based virtio-fs layer with a block device, letting the guest manage its own filesystem and eliminating metadata round trips. This enables block-level copy-on-write, bounding overhead to a single block instead of a full file copy.
- With ImageFS-blk, Modal achieved dramatic performance improvements over virtio-fs: chown dropped from 9.87s to 0.21s (47x faster), bun install from 12.5s to 1.07s (11.7x faster), and pgbench read-write throughput increased from 8,300 to 18,800 TPS (2.3x).

## FAQ

### Why was virtio-fs so slow for metadata operations?

Every metadata operation (e.g., chown) required multiple round trips from the guest to the host. A single chown decomposed into three FUSE requests: getxattr, getattr, and setattr, each requiring host syscalls to ImageFS. This added approximately 99 µs per chown and multiplied overhead for bulk operations.

### How does ImageFS-blk differ from the original ImageFS stack?

ImageFS-blk exposes a virtual block device to the guest VM rather than sharing a FUSE filesystem. The guest runs its own ext4 on this device, so all metadata interpretation happens inside the guest. The host only serves blocks on demand, caching them from ImageFS's tiered storage.

### What performance gains did ImageFS-blk deliver over virtio-fs?

In microbenchmarks, chown went from 9.87s to 0.21s, stat from 3.23s to 0.90s. In real workloads, bun install dropped from 12.5s to 1.07s, and ripgrep on a warm cache went from 6.03s to 0.18s. Database benchmarks showed similar improvements: SQLite speedtest1 runtime fell from 543.4s to 43.2s.

**Tags:** [Block Device](https://yomu.fyi/topic/block-device), [Filesystems](https://yomu.fyi/topic/filesystems), [virtio-fs](https://yomu.fyi/topic/virtio-fs), [Virtualization](https://yomu.fyi/topic/virtualization)

- Source: [Modal](https://modal.com/blog/vm-sandbox-filesystem)
- Source URL: https://modal.com/blog/vm-sandbox-filesystem
- Ingested by Yomu: 2026-10-16T18:00:17.386Z

[Read original post](https://modal.com/blog/vm-sandbox-filesystem)