---
title: "Observability"
description: "250 posts about Observability, summarised, each linking to the original."
---

# Observability
> 250 posts about Observability, summarised, each linking to the original.

## Articles

### [The product analytics you already have](https://yomu.fyi/post/the-product-analytics-you-already-have.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Rahul Chhabria
- Published: May 21, 2026
- Type: Explainer

Software engineers frequently rely on separate product analytics tools and teams to answer user behavior questions, creating disjointed data models and delayed insights. Existing debugging telemetry within tools like Sentry—specifically spans, structured logs, and application metrics—can serve as the primary source for product metrics. Developers enrich request spans with business-level attributes, record discrete business events through wide structured logs, and track KPIs with metric counters and distributions. This unified telemetry links business outcomes directly to underlying traces, errors, releases, and source code lines, enabling immediate root-cause analysis and automated alerts. While multi-session cohort and funnel aggregation remains a current tooling gap, standard telemetry solves day-to-day adoption, performance, and revenue questions.


### [May 2026 newsletter](https://yomu.fyi/post/may-2026-newsletter.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Mark Needham
- Published: May 21, 2026
- Type: Announcement

The May 2026 ClickHouse newsletter details enterprise observability migrations, ecosystem integrations, and core database enhancements. Digital banking platform Qonto migrated from Grafana Tempo to ClickHouse Cloud, compressing 231 TB of raw trace metadata to 376 GB while expanding query windows to two weeks and introducing an AI incident companion. Similarly, LINE MAN Wongnai consolidated multiple business clusters onto self-hosted ClickHouse, achieving 1.5 million rows per second peak ingest, 100% trace retention, and a 53% reduction in observability costs. Comparative log analytics benchmarks demonstrated that ClickHouse utilized 5x less disk space and achieved 4 to 6x faster cold query execution than Elasticsearch across 50 billion rows. Additional updates include the ClickHouse 26.4 release featuring improved SQL compatibility, new agent telemetry adapters in Mastra, and SQL-based alerting capabilities in ClickStack.


### [What's new in ClickStack - April 2026](https://yomu.fyi/post/what-s-new-in-clickstack-april-2026.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: The ClickStack Team
- Published: May 19, 2026
- Type: Announcement

ClickStack's April release introduces several core enhancements across querying, alerting, and dashboard visualization capabilities. The platform added SQL-powered alerting, allowing teams to express complex statistical rules like rolling baselines and anomaly detection using native ClickHouse SQL and window functions without external pipelines. To improve search and filtering performance across common observability workloads, the default otel\_logs schema was redesigned with ClickHouse text indexes and adjusted ordering keys. Furthermore, heatmaps transitioned into a first-class chart type inside the shared visualization system for latency distribution tracking. The update also delivers persistent alert execution error reporting, expanded threshold comparison operators, and per-series number formatting across dashboards.


### [What’s new in Unity AI Gateway: service policies, guardrails, observability, and cost controls for AI agents and MCPs](https://yomu.fyi/post/what-s-new-in-unity-ai-gateway-service-policies-guardrails-observabili.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: David Nasi, Kelly Albano
- Published: May 19, 2026
- Type: Announcement

Unity AI Gateway is expanding runtime governance for production AI agents, model calls, and MCP tool interactions as teams face rising costs, unclear behavior, and limited control. The Beta adds LLM-based guardrails, token-level cost attribution with per-user alerts and hard budget limits, payload logging, and MCP service policies. Guardrails use a model and prompt to evaluate inputs, outputs, or both in real time, while inference tables and Unity Catalog system tables centralize governed records of usage and interactions. For MCPs, administrators can define SQL policies as Unity Catalog functions using agent identity, user context, and request parameters to constrain tool access and sensitive actions. The capabilities are available in Beta, with payload logging and service policies offered through gated enrollment, and are intended to improve observability, compliance, and cost control.


### [How the D. E. Shaw group powers high-cardinality observability at scale with ClickHouse](https://yomu.fyi/post/how-the-d-e-shaw-group-powers-high-cardinality-observability-at-scale.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: ClickHouse
- Published: May 15, 2026
- Type: Explainer

Inside D. E. Shaw group, a global investment and technology development firm, millions of compute workloads run monthly across its high-performance computing infrastructure. To support capacity planning and resource allocation, it collects task-level metrics for requested and utilized compute, producing millions of distinct time series that its previous observability platform could not accommodate. Engineers benchmarked databases on identical hardware and datasets; ClickHouse ingested approximately 3.5 million samples per second on one node versus roughly 480,000 for a competing solution, while similar queries completed in seconds instead of timing out. They retained InfluxDB line protocol, batched messages into ClickHouse, and built disk-based tooling to transform and stream historical data; the resulting four-server NVMe cluster stores about 68 TB compressed and sustains 530k records per second on average. Materialized views support capacity-planning dashboards, while OpenTelemetry tracing extends the platform to other observability workloads, with SQL migration and data modeling requiring engineering effort.


### [ClickHouse vs Prometheus for High Cardinality, Part 2: Cardinality in ClickHouse](https://yomu.fyi/post/clickhouse-vs-prometheus-for-high-cardinality-part-2-cardinality-in-cl.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Rory Crispin, Dale McDiarmid
- Published: May 15, 2026
- Type: Explainer

Traditional time-series databases like Prometheus experience write amplification, memory overhead, and operational challenges when high-cardinality label combinations churn rapidly. ClickHouse alters these economics by abandoning dedicated series objects in favor of a wide events model, storing telemetry as rows in columnar tables with dynamic Map attributes and metric measurements. Dynamic attributes can be handled using sharded Map types alongside materialized columns and text indexes for frequently queried labels. This design trades ingestion overhead for query-time processing, relying on columnar compression, partition pruning, and parallel scans to compute aggregations on demand. ClickHouse avoids the ingest-side bottlenecks of high cardinality, though Prometheus remains preferable for moderate cardinality, native PromQL semantics, and single-series lookups.


### [ClickHouse vs Prometheus for High Cardinality, Part 1: Understanding the Problem](https://yomu.fyi/post/clickhouse-vs-prometheus-for-high-cardinality-part-1-understanding-the.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Rory Crispin, Dale McDiarmid
- Published: May 14, 2026
- Type: Explainer

Observability systems frequently encounter high cardinality when metrics contain numerous unique label combinations across multidimensional cloud infrastructure. In Prometheus, every unique label combination instantiates an independent time series that incurs metadata overhead, requires in-memory memSeries registration, and continually expands inverted index posting lists. The ingestion engine stores samples in two-hour Head blocks using compressed 120-sample chunks with XOR-based encodings, but rapid series creation and elevated scrape frequencies multiply chunk allocations and memory consumption. When executing read operations, broad aggregations and regular expressions require intersecting expansive posting lists and decoding entire chunks sequentially because the engine cannot push value predicates directly into compressed storage blocks. Consequently, managing high-cardinality workloads in series-oriented architectures forces operational trade-offs between system stability and granular visibility into ephemeral components.


### [New ways to agentically build and edit dashboards](https://yomu.fyi/post/new-ways-to-agentically-build-and-edit-dashboards.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Ben Coe, Steve Zegalia
- Published: May 14, 2026
- Type: Announcement

Pre-configured monitoring templates often fail to meet organization-specific needs, forcing developers to manually build dashboards widget by widget. Sentry introduced AI-agent-powered dashboard creation and editing in open beta, alongside CLI management tooling and template revisions. Users can prompt an AI chat interface to build or adjust dashboards, duplicate and customize uneditable Sentry-built templates, or execute dashboard commands directly in their terminal. Dashboards leverage issue, tracing, and application metric data across frontend, backend, mobile, and framework-specific views like Next.js and Laravel. Edits made through the UI, CLI, or AI agent are automatically tracked in a revision history that supports one-click rollbacks.


### [ClickStack SQL Charting and Alerting](https://yomu.fyi/post/clickstack-sql-charting-and-alerting.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Drew Davis, Dale McDiarmid
- Published: May 13, 2026
- Type: Announcement

ClickStack introduced SQL-based visualizations and SQL-based alerting to allow users to build charts and alerts using arbitrary ClickHouse SQL queries. Traditional query builders trade analytical flexibility for simplicity, preventing teams from easily expressing multi-stage computations like rolling baselines, anomaly detection, or dynamic SLO calculations. To maintain interactivity within dashboards, the system supports ClickHouse query parameters and macros that inject active time ranges, interval sizes, dashboard filters, and source tables. Furthermore, SQL-based alerting evaluates the numeric columns output by these queries, such as returning binary flags when values exceed statistical boundaries. By shifting computational logic directly into SQL, teams can implement complex operational checks without relying on static external thresholds.


### [Building Blocks for Foundation Model Training and Inference on AWS](https://yomu.fyi/post/building-blocks-for-foundation-model-training-and-inference-on-aws.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Keita Watanabe, Pavel Belevich, Aman Shanbhag
- Published: May 11, 2026
- Type: Explainer

The post explains how foundation-model scaling now spans pre-training, post-training, and test-time compute, creating shared infrastructure requirements rather than a single pre-training curve. It examines how AWS infrastructure and open-source tools support these workloads across accelerated compute, networking, storage, orchestration, ML frameworks, and observability. Technical detail includes NVIDIA H100, H200, B200, and B300 GPU characteristics, NVLink/NVSwitch scale-up communication, and EFA-based scale-out networking for distributed collectives. The architecture also incorporates EC2 P instances, tiered storage, Slurm, Kubernetes, SageMaker HyperPod, CUDA, NCCL, PyTorch, Prometheus, and Grafana. The conclusion presents integration points between these layers as the basis for diagnosing bottlenecks and making scaling decisions across the foundation-model lifecycle.


### [From vibe code to production-ready: observability for Next.js and Supabase apps](https://yomu.fyi/post/from-vibe-code-to-production-ready-observability-for-next-js-and-supab.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Sergiy Dybskiy
- Published: May 11, 2026
- Type: Problem & solution

AI-assisted software development often generates functional code that lacks proper instrumentation, leading to unindexed database queries, N+1 patterns, and fragmented connection strategies. While Supabase provides built-in query analytics, security advisors, and subsystem logs, it cannot trace distributed requests across the entire application stack. Integrating Sentry bridges this visibility gap by establishing distributed traces across Next.js frontends, Deno-based Supabase Edge Functions, and Postgres databases. Teams can forward Supabase logs into dedicated Sentry projects and configure runtime integrations to automatically detect performance bottlenecks such as slow spans and Web Vitals regressions. Furthermore, integrating Model Context Protocol servers allows Sentry's Seer debugger to identify root causes and generate automated fixes for emerging issues.


### [Monitor Unreal Engine Game Performance with Application Metrics](https://yomu.fyi/post/monitor-unreal-engine-game-performance-with-application-metrics.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Ivan Tustanivskyi
- Published: May 8, 2026
- Type: Announcement

Unreal Engine games frequently experience performance issues such as frame-rate drops, hitches, and network degradation that do not register as application crashes. To address the lack of live production telemetry, the Sentry Unreal SDK introduces automatic performance metrics. The integration periodically samples frame times, network health, active UObject counts, physical memory usage, and garbage collection pause durations across desktop, console, and Android platforms. Telemetry entries attach contextual hardware and environment attributes alongside trace IDs to connect aggregate latency spikes directly to underlying traces and spans. Enabled by default in Unreal SDK version 1.11.0, this telemetry provides developers with fleet-wide visibility into hardware-specific bottlenecks across game releases.


### [Introducing Postgres Query Insights in ClickHouse Cloud](https://yomu.fyi/post/introducing-postgres-query-insights-in-clickhouse-cloud.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Amog Iska
- Published: May 7, 2026
- Type: Announcement

ClickHouse Cloud introduced Query Insights in preview for Managed Postgres to help teams diagnose slow database queries. Powered by the open-source extension pg\_stat\_ch, the tool streams per-statement telemetry directly into ClickHouse to enable fast aggregations across billions of execution rows. Postgres normalizes query literals into placeholders during the parse-analyze phase before telemetry leaves the database, preventing sensitive data exposure. Telemetry emission maintains around three percent producer overhead by utilizing a non-blocking try-lock on a shared-memory ring buffer. Users can examine high-level database health metrics, rank query patterns by runtime or latency percentiles, and inspect execution flyouts detailing disk spills, cache hits, and parallel worker behavior.


### [Fixing JavaScript observability, one library at a time](https://yomu.fyi/post/fixing-javascript-observability-one-library-at-a-time.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Abdelrahman Awad
- Published: May 7, 2026
- Type: Problem & solution

JavaScript application performance monitoring tools rely on monkey-patching via require-in-the-middle and import-in-the-middle, which breaks with ECMAScript Modules, bundlers, and non-Node runtimes. To solve this, an initiative is replacing monkey-patching with Node's built-in diagnostics\_channel TracingChannel API across server-side JavaScript libraries. Under this pattern, libraries publish structured events while monitoring vendors subscribe to them with zero overhead when unlistened. The author leveraged Claude Code workflows alongside direct maintainer communication to draft proposals, implement code, and manage reviews across 44 target packages. Ten libraries including mysql2, node-redis, ioredis, and unjs modules have merged support, while active efforts continue on shared OpenTelemetry mapper registries and remaining ecosystem packages.


### [Improved debugging for Expo apps with the React Native SDK](https://yomu.fyi/post/improved-debugging-for-expo-apps-with-the-react-native-sdk.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Aleksandr Pantiukhov
- Published: May 6, 2026
- Type: Announcement

Expo applications generate approximately 75% of the total event volume received from React Native applications in Sentry. In response, Sentry updated its React Native SDK starting in version 8.10 to improve Expo debugging and performance monitoring workflows. The update automatically enriches events with Expo constants and over-the-air update context, including channels, runtime versions, and emergency launch warnings when an OTA update fails. Developers can track remote EAS Build lifecycle failures and successes using dedicated hooks configured in package.json. Performance instrumentation now supports span generation for Expo Router route prefetching as well as asset and image loading through wrapper functions around expo-image and expo-asset.


### [The AI scaling gap hiding in digital native companies](https://yomu.fyi/post/the-ai-scaling-gap-hiding-in-digital-native-companies.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Madelyn Mullen
- Published: May 5, 2026
- Type: Explainer

An Economist survey of 1,220+ executives across eight industries finds that digital native companies lead in ambition and breadth of AI deployment, but not in full operational maturity. At 18%, they are the most likely group to prioritize embedding AI across core processes at scale, and nearly 92% report AI ROI ahead of plan. Yet they lead fully embedded AI—defined as use by 100+ users, SLAs, and performance and impact monitoring—in only one of eight functions, R&D/product development; they rank seventh in finance and sixth in operations and supply chain. Telecom, media and entertainment, manufacturing, and energy outperform them in selected functions despite lower stated scaling priority. The proposed response is to build shared, production-grade foundations for data, governance, workloads, models, agents, and applications, with security, lineage, monitoring, and performance measurement treated as reusable capabilities.


### [10 trillion samples a day: Scaling beyond traditional monitoring infra at Databricks](https://yomu.fyi/post/10-trillion-samples-a-day-scaling-beyond-traditional-monitoring-infra.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: David Yuan, Yi Jin, Karan Bavishi, HC Zhu, Joey Beyda
- Published: May 5, 2026
- Type: Problem & solution

Databricks’ monitoring infrastructure now tracks 5 billion active timeseries in real time and ingests more than 10 trillion samples daily, exposing scalability, reliability, cost, and operability limits in its older stack. The replacement, Pantheon, is a fork of CNCF Thanos deployed across more than 160 instances in about 70 regions and three cloud providers; tiered storage, differentiated memory retention, isolated replicated Receive groups, multitenancy, and a custom control plane support automated scaling and recovery. Pantheon’s largest instance holds about 300 million in-memory timeseries and handles nearly 1,000 PromQL queries per second, while migration reduced annual cloud costs by millions and monitoring downtime by roughly five times. For high-cardinality troubleshooting, Hydra preserves raw metrics in Delta tables, exposes them through Grafana and SQL, and unifies metric semantics across aggregated and raw paths, with freshness improvements planned.


### [Stop guessing in production: Full fidelity tracing at scale with ClickHouse and Odigos](https://yomu.fyi/post/stop-guessing-in-production-full-fidelity-tracing-at-scale-with-clickh.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Will Searle
- Published: May 5, 2026
- Type: Problem & solution

Distributed tracing frequently suffers from visibility gaps, broken context across asynchronous boundaries like Kafka, and high operational overhead from manually configuring OpenTelemetry SDKs and collectors. To resolve these issues, Odigos provides zero-code instrumentation via eBPF probes that dynamically attach to running workloads without modifying application code. An intelligent control plane within Odigos dynamically adjusts trace sampling and attribute enrichment based on real-time latency and error rates. The resulting telemetry exports over OTLP into ClickStack, an all-in-one backend featuring an OpenTelemetry Collector, ClickHouse storage, and the HyperDX query interface. This unified pipeline preserves end-to-end trace continuity, captures deep application and database context, and enables fast SQL-based trace analysis.


### [Introducing Application Metrics: Track the signal, see the spike, jump to the trace](https://yomu.fyi/post/introducing-application-metrics-track-the-signal-see-the-spike-jump-to.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Ben Coe
- Published: May 5, 2026
- Type: Announcement

Sentry launched Application Metrics to capture application signals without pre-aggregating counters or discarding high-cardinality attributes like user IDs, project IDs, and regions. The system supports counters, distributions, and gauges within the existing SDK while linking each metric event directly to trace IDs for integrated debugging. Engineers demonstrated the feature by troubleshooting a Session Replay failure that triggered silently when sessions exceeded 1,000 video segments. By instrumenting distributions for replay event counts with project and replay attributes, the team isolated seven affected occurrences, navigated to connected traces and log lines, and reproduced the issue. Application Metrics comes enabled in recent SDK versions with 5GB included per plan.


### [Goodbye limitations, hello data: How Qonto is rethinking observability with ClickHouse Cloud](https://yomu.fyi/post/goodbye-limitations-hello-data-how-qonto-is-rethinking-observability-w.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: ClickHouse Team
- Published: May 1, 2026
- Type: Problem & solution

Qonto struggled with slow queries, system instability, and aggressive data sampling when handling distributed traces in Grafana Tempo. To enable wide-event querying without strict cardinality limits, the team migrated their telemetry storage to ClickHouse Cloud. Telemetry from applications, Kubernetes, and GitHub routes through OpenTelemetry collectors over AWS PrivateLink into ClickHouse, while Grafana remains the primary dashboard interface. By compressing 231 TB of uncompressed trace attributes down to 376 GB, Qonto achieved substantial storage savings and integrated an AI incident companion using the ClickHouse Model Context Protocol. The new architecture eliminated cardinality policing, allowed product and support teams to self-serve incident investigations, and enabled real-time streaming pre-aggregations using Apache Flink.


[Newer posts](https://yomu.fyi/topic/observability/page/7.md) · [Older posts](https://yomu.fyi/topic/observability/page/9.md)