---
title: "Fly Behind The Scenes: Fresh Logging"
description: "Fly replaced its centralized Graylog logging platform after seven servers could not process peak volumes of 20,000–30,000 logs per second, causing drops. Vector, written in Rust, now runs on every server, parses and transforms logs from journald and other applications, and sends them directly to Elasticsearch, removing Graylog as a middleman. The pipeline adopts the Elasticsearch Common Schema (ECS), with Fly-specific fields for app names, allocation IDs, and regions, while Vector configurations are distributed through the existing configuration management system. The migration took about two weeks alongside other work; logs are now available quickly enough to support failed-allocation queries that previously often raced processing, and future sinks may include Papertrail, Honeycomb, or Kafka."
---

# Fly Behind The Scenes: Fresh Logging

[Fly.io](https://yomu.fyi/company/fly-io) · Oct 15, 2020

**Type:** Problem & solution

## Summary

Fly replaced its centralized Graylog logging platform after seven servers could not process peak volumes of 20,000–30,000 logs per second, causing drops. Vector, written in Rust, now runs on every server, parses and transforms logs from journald and other applications, and sends them directly to Elasticsearch, removing Graylog as a middleman. The pipeline adopts the Elasticsearch Common Schema (ECS), with Fly-specific fields for app names, allocation IDs, and regions, while Vector configurations are distributed through the existing configuration management system. The migration took about two weeks alongside other work; logs are now available quickly enough to support failed-allocation queries that previously often raced processing, and future sinks may include Papertrail, Honeycomb, or Kafka.

## Context

The previous logging system used seven centralized Graylog servers that could not process peak volumes of 20,000–30,000 logs per second, causing dropped logs. Customer access also depended on polling the Graylog API, while failed-allocation queries could run before logs had been processed.

## Approach / What changed

Vector runs on every server, receives logs from journald and other applications, applies parsing and transformations, and ships the results directly to Elasticsearch. The platform uses the Elasticsearch Common Schema with Fly-specific fields, and Vector TOML configurations are distributed through the existing configuration management system.

## Takeaways

- Vector distributes log capture and transformation across servers, while Elasticsearch handles indexing and retrieval after Graylog was removed from the path.
- The Elasticsearch Common Schema standardizes fields across applications, including source IP data, and supports Fly-specific fields such as app names, allocation IDs, and regions.
- Vector configurations can be validated and unit tested locally before deployment; configuration changes are then distributed to all servers through the configuration management system.

**Tags:** [Elasticsearch](https://yomu.fyi/topic/elasticsearch), [Rust](https://yomu.fyi/topic/rust), [Vector](https://yomu.fyi/topic/vector)

- Source: [Fly.io](https://fly.io/blog/fly-behind-the-scenes-fresh-logging)
- Source URL: https://fly.io/blog/fly-behind-the-scenes-fresh-logging
- Ingested by Yomu: 2026-09-02T12:00:38.776Z

[Read original post](https://fly.io/blog/fly-behind-the-scenes-fresh-logging)