---
title: "Multiple Logs for Resiliency"
description: "An unexpected network outage can make an application misbehave while also removing crucial log data from the normal Fly.io pipeline. The post traces that pipeline from application STDOUT through Fly.io, NATS, the Log Shipper, Vector, and a third-party provider, then presents two ways to add redundancy. One strategy broadcasts framework logs to a separate file on each machine, preferably on a persistent volume with rotation; a Rails example uses ActiveSupport::Logger.broadcast. The other runs a separate app that subscribes directly to NATS, parses messages, and writes selected fields to disk, with a Fly authentication token and provisions for reconnection and error handling. File copies are more failsafe but raw and harder to fetch, while a custom shipper can produce more usable data but is less bulletproof."
---

# Multiple Logs for Resiliency

[Fly.io](https://yomu.fyi/company/fly-io) · Jul 21, 2023

**Type:** Problem & solution

## Summary

An unexpected network outage can make an application misbehave while also removing crucial log data from the normal Fly.io pipeline. The post traces that pipeline from application STDOUT through Fly.io, NATS, the Log Shipper, Vector, and a third-party provider, then presents two ways to add redundancy. One strategy broadcasts framework logs to a separate file on each machine, preferably on a persistent volume with rotation; a Rails example uses ActiveSupport::Logger.broadcast. The other runs a separate app that subscribes directly to NATS, parses messages, and writes selected fields to disk, with a Fly authentication token and provisions for reconnection and error handling. File copies are more failsafe but raw and harder to fetch, while a custom shipper can produce more usable data but is less bulletproof.

## Context

A network outage can cause application problems while crucial logs are lost through the normal chain from STDOUT to Fly.io, NATS, the Log Shipper, Vector, and a third-party logging provider. The post considers how to retain log data when that pipeline fails.

## Approach / What changed

The post proposes adding one or both of two redundant logging strategies: writing application logs to per-machine files on persistent volumes, or operating a separate app that subscribes directly to NATS and stores selected log fields. It also covers log rotation, Fly authentication, reconnection, filtering, and write-error handling.

## Takeaways

- Per-machine log files provide the most failsafe redundancy, but they are raw and must be fetched from each machine; persistent volumes and rotation help preserve them without filling available storage.
- A Rails implementation can use ActiveSupport::Logger.broadcast to send entries to STDOUT and a separate production log file.
- A custom shipper can subscribe to the logs.> NATS subject, parse JSON messages, and write timestamp, instance, region, level, and message fields to disk, but it needs safeguards for write failures and lost NATS connections.

**Tags:** [Fly](https://yomu.fyi/topic/fly), [NATS](https://yomu.fyi/topic/nats), [Reliability](https://yomu.fyi/topic/reliability)

- Source: [Fly.io](https://fly.io/blog/redundant-logs)
- Source URL: https://fly.io/blog/redundant-logs
- Ingested by Yomu: 2026-09-02T12:00:07.250Z

[Read original post](https://fly.io/blog/redundant-logs)