Structured Logging in .NET with Serilog and ClickHouse

Summary

Traditional logging solutions like Seq, ELK, and cloud platforms often face scaling bottlenecks, including slower queries, higher operational costs, and complex pipeline management. To address this, an ASP.NET service can write structured logs directly to ClickHouse using Serilog and the Serilog.Sinks.ClickHouse library. The sink transmits data over HTTP and provides a fluent C# builder to configure table schemas, typed JSON columns, full-text indexes, and MergeTree engine parameters. ClickHouse stores log data in a columnar format that delivers 10 to 20 times compression while enabling direct SQL queries, JSON property filtering, and real-time aggregations through materialized views. Furthermore, the sink includes an in-memory buffer to handle transient network drops with configurable batching, retry mechanisms, and callback hooks for failure alerting.

Takeaways

1. 1

   Serilog.Sinks.ClickHouse writes directly to ClickHouse via HTTP, allowing complete C# schema configuration including typed JSON subcolumns and text indexes.

2. 2

   ClickHouse achieves 10x to 20x compression on structured logs and enables SQL-based querying, filtering, and real-time aggregation via materialized views.

3. 3

   The Serilog ClickHouse sink buffers log events in memory and retries during flush cycles to handle brief network interruptions.