---
title: "Corrosion"
description: "Corrosion is Fly.io’s open-source service discovery and state synchronization system, built to distribute routing data across globally deployed edge proxies and worker servers. It replaces centralized consensus-oriented designs with a Rust program that propagates a SQLite database through SWIM-based gossip and QUIC, using cr-sqlite’s CRDT tables and logical timestamps to reconcile updates without locking, a central server, or distributed consensus. The system’s design follows link-state routing ideas and exploits Fly.io’s fully connected WireGuard mesh, while worker-owned state keeps conflicts uncommon. The post also documents failures, including a contagious Rust RWLock deadlock, a nullable-column migration that triggered large-scale reconciliation, and the need for backups, whole-dataset republishing, testing, and regionalization to reduce blast radius. Corrosion now converges in seconds across thousands of workers."
---

# Corrosion

[Fly.io](https://yomu.fyi/company/fly-io) · Oct 22, 2025

**Type:** Explainer

## Summary

Corrosion is Fly.io’s open-source service discovery and state synchronization system, built to distribute routing data across globally deployed edge proxies and worker servers. It replaces centralized consensus-oriented designs with a Rust program that propagates a SQLite database through SWIM-based gossip and QUIC, using cr-sqlite’s CRDT tables and logical timestamps to reconcile updates without locking, a central server, or distributed consensus. The system’s design follows link-state routing ideas and exploits Fly.io’s fully connected WireGuard mesh, while worker-owned state keeps conflicts uncommon. The post also documents failures, including a contagious Rust RWLock deadlock, a nullable-column migration that triggered large-scale reconciliation, and the need for backups, whole-dataset republishing, testing, and regionalization to reduce blast radius. Corrosion now converges in seconds across thousands of workers.

## Context

Fly.io needed a globally synchronized routing map because its orchestration model makes individual worker servers authoritative for their workloads. Existing approaches based on Consul, SQLite caches, and distributed consensus imposed costs or risks that did not fit a globally distributed platform.

## Approach / What changed

Corrosion is a Rust service that propagates SQLite databases over a fully connected WireGuard mesh. It uses SWIM for membership, QUIC for broadcasting and reconciliation, and cr-sqlite CRDT tables with logical timestamps to converge state without locking, central servers, or distributed consensus. Later changes added checkpoint backups, whole-dataset republishing, extensive testing, and regionalization.

## Takeaways

- Corrosion combines SWIM membership gossip with QUIC update distribution and cr-sqlite CRDT tables, which apply changes using last-write-wins logical timestamps.
- Worker-owned state avoids bottlenecking on a globally consistent central scheduler because updates from different workers almost never conflict.
- Regionalization separates fine-grained regional clusters from a global application-to-region map, reducing the blast radius of state-related bugs; checkpoint backups support rebuilding the database.

**Tags:** [Distributed Systems](https://yomu.fyi/topic/distributed-systems), [Rust](https://yomu.fyi/topic/rust), [SQLite](https://yomu.fyi/topic/sqlite), [WireGuard](https://yomu.fyi/topic/wireguard)

- Source: [Fly.io](https://fly.io/blog/corrosion)
- Source URL: https://fly.io/blog/corrosion
- Ingested by Yomu: 2026-09-03T01:24:42.897Z

[Read original post](https://fly.io/blog/corrosion)