---
title: "How Trigger.dev is using ClickHouse to scale observability for long-running AI workflows"
description: "Trigger.dev, an open-source platform for serverless asynchronous tasks, needed to scale observability for increasingly complex, long-running AI workflows spanning thousands of tenants. Its initial Postgres design stored logs, traces, and events in a growing table, making indexes, analytics, and operational reliability increasingly difficult to manage. The team moved observability workloads to ClickHouse after evaluating performance, scalability, open-source compatibility, deployment flexibility, and support for dynamic telemetry, including its native JSON data type. Migration occurred gradually at the task-execution level, routing each run’s logs to either database without downtime or customer intervention. ClickHouse now supports real-time telemetry analysis, approximately 200-millisecond P95 latency for queries returning tens of thousands of rows, and a dramatically smaller telemetry storage footprint."
---

# How Trigger.dev is using ClickHouse to scale observability for long-running AI workflows

[Clickhouse](https://yomu.fyi/company/clickhouse) · Jul 9, 2026

**Type:** Problem & solution

## Summary

Trigger.dev, an open-source platform for serverless asynchronous tasks, needed to scale observability for increasingly complex, long-running AI workflows spanning thousands of tenants. Its initial Postgres design stored logs, traces, and events in a growing table, making indexes, analytics, and operational reliability increasingly difficult to manage. The team moved observability workloads to ClickHouse after evaluating performance, scalability, open-source compatibility, deployment flexibility, and support for dynamic telemetry, including its native JSON data type. Migration occurred gradually at the task-execution level, routing each run’s logs to either database without downtime or customer intervention. ClickHouse now supports real-time telemetry analysis, approximately 200-millisecond P95 latency for queries returning tens of thousands of rows, and a dramatically smaller telemetry storage footprint.

## Context

Trigger.dev’s observability data grew as customers used the platform for long-running AI workflows and complex execution chains. Postgres struggled with the volume and dynamic structure of logs, traces, and events, while analytical workloads risked affecting the operational database that ran customer tasks.

## Approach / What changed

Trigger.dev migrated observability workloads from Postgres to ClickHouse incrementally, using individual task executions, called runs, as the routing unit. During the transition, each run’s logs could be written to either database, allowing production validation without downtime. The team selected ClickHouse for performance, scalability, SQL familiarity, open-source compatibility, flexible deployment, and support for dynamic telemetry.

## Takeaways

- Postgres telemetry was partitioned and optimized, but growing indexes, slower queries, and analytical workloads continued to create operational and scalability risks.
- ClickHouse enabled Trigger.dev to analyze logs, traces, events, and usage patterns in real time without placing those analytical workloads on the operational database.
- Queries retrieving tens of thousands of rows achieve around 200 milliseconds P95 latency, while the telemetry storage footprint was dramatically reduced.

**Tags:** [Monitoring](https://yomu.fyi/topic/monitoring), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance), [Postgres](https://yomu.fyi/topic/postgres), [Scalability](https://yomu.fyi/topic/scalability)

- Source: [Clickhouse](https://clickhouse.com/blog/trigger-dev-scaling-observability)
- Source URL: https://clickhouse.com/blog/trigger-dev-scaling-observability
- Ingested by Yomu: 2026-08-29T01:38:11.140Z

[Read original post](https://clickhouse.com/blog/trigger-dev-scaling-observability)