---
title: "Operationalizing Macaroons"
description: "Operationalizing Macaroons at Fly.io requires treating them as online-stateful bearer tokens whose nonces must be checked against database state, while keeping root secrets away from the primary API cluster. The team built tkdb, a roughly 5,000-line Go service backed by encrypted SQLite managed with LiteFS and Litestream, deployed on isolated hardware across the US, Europe, and Australia. Its HTTP/Noise interface supports verification and restricted signing, while FlyCast routing, retries, and client-side caching reduce the impact of unreliable global connectivity; verification caching exceeds 98%. Revocation uses blacklist state, and third-party caveats let services such as flyd obtain narrowly scoped secret-reading authority through tkdb rather than holding broad credentials. OpenTelemetry, standard metrics, and a retained OpenSearch audit trail provide visibility, while the post concludes that Macaroons’ internal infrastructure benefits outweigh limited user adoption."
---

# Operationalizing Macaroons

[Fly.io](https://yomu.fyi/company/fly-io) · Mar 27, 2025

**Type:** Explainer

## Summary

Operationalizing Macaroons at Fly.io requires treating them as online-stateful bearer tokens whose nonces must be checked against database state, while keeping root secrets away from the primary API cluster. The team built tkdb, a roughly 5,000-line Go service backed by encrypted SQLite managed with LiteFS and Litestream, deployed on isolated hardware across the US, Europe, and Australia. Its HTTP/Noise interface supports verification and restricted signing, while FlyCast routing, retries, and client-side caching reduce the impact of unreliable global connectivity; verification caching exceeds 98%. Revocation uses blacklist state, and third-party caveats let services such as flyd obtain narrowly scoped secret-reading authority through tkdb rather than holding broad credentials. OpenTelemetry, standard metrics, and a retained OpenSearch audit trail provide visibility, while the post concludes that Macaroons’ internal infrastructure benefits outweigh limited user adoption.

## Context

Macaroon verification is online-stateful because each token’s nonce must be looked up in a database. Fly.io wanted to keep that state and the root secrets used to protect it out of the primary API cluster because API instability could affect deployed applications, and because secure design favors isolating sensitive secrets from complicated code. Global connectivity problems also make transoceanic verification links undesirable.

## Approach / What changed

Fly.io created tkdb, a deliberately simple Go service managing encrypted SQLite with LiteFS and Litestream on isolated hardware in the US, Europe, and Australia. It exposes verification and signing APIs over HTTP/Noise, uses FlyCast routing and retry behavior, and relies on client-side verification caching. Revocation is stored in database state, while third-party caveats and narrowly distributed signing credentials minimize service privileges. OpenTelemetry, Prometheus metrics, and a permanently retained OpenSearch audit trail support operations and investigation.

## Takeaways

- tkdb is approximately 5,000 lines of Go and uses encrypted SQLite managed by LiteFS and Litestream; the database is only a couple dozen megabytes after several years of Macaroon use.
- Macaroon verification caching exceeds 98% because a verified token contains enough information to verify more-specific descendants produced through the chaining HMAC construction.
- flyd receives a secret-reading Macaroon with a third-party caveat, requiring tkdb to verify organization permissions instead of granting every worker broad access to user secrets.

**Tags:** [LiteFS](https://yomu.fyi/topic/litefs), [Litestream](https://yomu.fyi/topic/litestream), [Macaroons](https://yomu.fyi/topic/macaroons), [SQLite](https://yomu.fyi/topic/sqlite)

- Source: [Fly.io](https://fly.io/blog/operationalizing-macaroons)
- Source URL: https://fly.io/blog/operationalizing-macaroons
- Ingested by Yomu: 2026-09-02T11:56:47.607Z

[Read original post](https://fly.io/blog/operationalizing-macaroons)