Shipping Logs

Summary

Shipping Logs describes how Fly.io application stdout moves through the platform and how to centralize and retain it beyond Fly.io’s log lifetime. Apps run in Firecracker VMs, where an injected Rust init process captures stdout and sends it through host-side Go code to Vector, which publishes the logs to an internal NATS cluster; Flaps limits access to an organization’s logs. The Fly Log Shipper runs Vector as a NATS client and forwards organization-wide streams to a configured sink such as Logtail, Loki, Datadog, or Cloudwatch, using secrets for provider credentials. Deployment involves launching the image, removing the generated services section, adding an internal port, and running fly deploy. A SUBJECT environment variable can narrow collection with NATS subjects such as logs.sandwich.>, covering an app across regions and instances.

Takeaways

1. 1

   The log path runs from application stdout through the VM’s Rust init process, host-side Go code, Vector, NATS, and the Flaps access proxy.

2. 2

   The Fly Log Shipper can forward organization-wide logs to sinks including Logtail, Loki, Datadog, and Cloudwatch, with provider configuration supplied through secrets.

3. 3

   NATS subjects can filter logs by application; logs.sandwich.> matches the sandwich app across regions and instances.