---
title: "Monitoring"
description: "198 posts about Monitoring, summarised, each linking to the original."
---

# Monitoring
> 198 posts about Monitoring, summarised, each linking to the original.

## Articles

### [What's new in Postgres Managed by ClickHouse: RBAC, Terraform, ClickPipes, extensions, and more](https://yomu.fyi/post/what-s-new-in-postgres-managed-by-clickhouse-rbac-terraform-clickpipes.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 18, 2026
- Type: Announcement

Postgres managed by ClickHouse, which entered public beta in May, received a broad update covering console operations, per-service RBAC, Terraform, Stripe provisioning, ClickPipes migrations, billing, capacity reservations, and extensions. The redesigned console adds service health, resource, and query views, while RBAC provides View and Manage roles across the console and public REST API; the alpha Terraform provider covers lifecycle operations including resizing, HA, replicas, restore, and password rotation. Postgres-to-Postgres ClickPipes now automate schema migration, usage is integrated into ClickHouse Cloud’s metering and billing, and AWS capacity reservations are allocated dynamically across availability zones. Underneath, pg\_clickhouse and pg\_stat\_ch moved to the header-only clickhouse-c library, removing a C++ exception conflict, using Postgres memory contexts, and reducing the x86-64 Linux extension binary from 4.9 MB to 1.4 MB; the update shipped less than a month after beta, with Terraform and extensions still evolving.


### [Payment Fraud Detection: How Banks and Businesses Stop Fraudulent Transactions](https://yomu.fyi/post/payment-fraud-detection-how-banks-and-businesses-stop-fraudulent-trans.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Jun 17, 2026
- Type: Explainer

Payment fraud detection is presented as a real-time practice for identifying and blocking unauthorized transactions amid evolving card-not-present, account takeover, authorised push payment, friendly, and card-testing attacks. Modern systems combine predefined rules, machine learning, behavioral analytics, device fingerprints, geolocation, transaction history, and velocity signals to calculate risk scores within milliseconds, declining or routing high-risk payments to review. Rules are fast and interpretable but require continual tuning, while machine-learning and behavioral models can identify complex or user-specific patterns that static thresholds miss. The source also emphasizes prevention controls such as MFA, tokenization, 3D Secure, and device binding, alongside unified real-time data infrastructure that supports cross-channel signals, continuous retraining, and deployment into payment authorization flows.


### [GCP Pub/Sub connector for ClickPipes is now in Private Preview](https://yomu.fyi/post/gcp-pub-sub-connector-for-clickpipes-is-now-in-private-preview.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 17, 2026
- Type: Announcement

ClickHouse has announced a GCP Pub/Sub connector for ClickPipes, now in Private Preview, to stream GCP infrastructure logs, application events, and operational signals into ClickHouse Cloud in real time. Previously, routing data from Pub/Sub into ClickHouse required GCS or a Dataflow pipeline; the connector instead manages subscription lifecycle, schema inference, and message delivery after a topic and target table are configured. It supports JSON, Avro, and Protobuf, Pub/Sub Schema Registry integration, automatic decompression, attribute-based filtering, and start positions of latest, earliest, or a specific timestamp. ClickPipes supports horizontal and vertical scaling, configurable replicas, automatic retries, built-in metrics and logs, and programmatic management through OpenAPI and the ClickHouse Terraform Provider. The connector is available upon request and free during Private Preview, with access offered through a waitlist or a ClickHouse Cloud account manager.


### [Introducing Genie ZeroOps: Put your data and AI operations on autopilot](https://yomu.fyi/post/introducing-genie-zeroops-put-your-data-and-ai-operations-on-autopilot.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Bilal Aslam, Lennart Kats, Ray Zhu, Mike Del Balso, Ori Zohar
- Published: Jun 16, 2026
- Type: Announcement

Genie ZeroOps is an autonomous background agent for monitoring and operating data and AI assets, including jobs, pipelines, tables, and ML models. It continuously detects visible and silent failures, uses Unity Catalog lineage and platform observability to assess root causes, generates remediation through development workflows, and verifies fixes in isolated sandboxes. These environments use shallow, zero-copy table clones, scoped permissions, and network isolation, so proposed changes run against real data without touching production or applying anything before approval. For ML workloads, the agent can diagnose degraded predictions, train a candidate on corrected features, evaluate it against the production model’s existing eval suite and criteria, and support live-traffic ramping when it is measurably better. Genie ZeroOps is entering private preview in the coming weeks, initially supporting jobs, pipelines, tables, and ML workloads; Apps and Lakebase databases are on the roadmap.


### [How QRT powers real-time research and risk management at petabyte scale](https://yomu.fyi/post/how-qrt-powers-real-time-research-and-risk-management-at-petabyte-scal.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 9, 2026
- Type: Explainer

Qube Research & Technologies (QRT), a global quantitative investment manager headquartered in London, uses ClickHouse Cloud for a data platform that serves researchers and for near-real-time risk monitoring, management, and P&L. Founded in 2018 as a cloud-native firm after the spinout of Credit Suisse’s proprietary trading arm, QRT chose a cloud-based approach rather than inheriting decades of on-premise infrastructure. The platform supports two major systems, while ClickStack consolidates logs, metrics, and traces for observability; alerts, log queries, and metric aggregations run against a single ClickHouse instance using standard SQL. QRT says its previous database was constrained by writer and reader counts and total cluster load, whereas ClickHouse provides greater speed and scalability, and that a unified monitoring backend simplifies maintenance while remaining reliably fast.


### [Monitor SLAs and scale ClickHouse Cloud with clickhousectl and agents](https://yomu.fyi/post/monitor-slas-and-scale-clickhouse-cloud-with-clickhousectl-and-agents.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 5, 2026
- Type: Tutorial

ClickHouse Cloud can automatically scale infrastructure up and down, but teams may need to monitor latency SLAs for specific workloads such as a frontend dashboard requiring p99 under 200 ms. The guide tags queries with log\_comment, reads system.query\_log through clickhousectl, and aggregates query\_duration\_ms over a defined window to measure the SLA. When it is breached, operators compare query volume and latency with Prometheus-compatible CPU, memory, and concurrency metrics to distinguish increased concurrency from resource contention, then choose horizontal replica scaling or vertical replica sizing. A cron-based script can scale out when p99 exceeds a threshold, while an agent can inspect evidence and apply one scaling action using clickhousectl; actions are recorded in the ClickHouse Cloud activity log.


### [Errors, traces, logs, metrics: when to reach for what](https://yomu.fyi/post/errors-traces-logs-metrics-when-to-reach-for-what.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Sergiy Dybskiy
- Published: Jun 5, 2026
- Type: Explainer

Modern application telemetry relies on four overlapping signals: errors, traces, logs, and metrics. Each telemetry type addresses distinct engineering questions and powers specific debugging workflows. Errors track grouped, actionable application crashes, whereas traces map timing and execution paths across dependency waterfalls. Metrics aggregate numerical measurements across historical deployments, and logs preserve complete state and decision-making context at specific moments in execution. Emitting telemetry directly into the format needed for a given workflow proves superior to relying on a single wide event, as tools like AI coding agents require structured spans to optimize execution paths while human debuggers rely on unsampled logs and metrics to pinpoint silent failures.


### [Query Tags: The Context Your Warehouse Queries Have Been Missing](https://yomu.fyi/post/query-tags-the-context-your-warehouse-queries-have-been-missing.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: JooHo Yeo, Jiabin Hu
- Published: Jun 2, 2026
- Type: Announcement

Databricks SQL Query Tags address a visibility gap in warehouse workloads: built-in logs identify users, warehouses, and tools, but not dashboards, projects, teams, or cost centers. Query Tags, now in Public Preview, attach multiple custom key-value pairs to each SQL execution, carry them into the Query History System Table, and expose them in the Query Profile UI. Partner integrations can automatically tag dbt models, while Power BI and Tableau support connection-level tags, and APIs and connectors support connection- or statement-level metadata. Users can also issue SET QUERY\_TAGS in SQL Editor, notebooks, dashboards, or alerts to label subsequent session statements. These tags enable SQL or Genie queries for cost allocation, regression analysis, workload filtering, and environment comparisons; future plans include broader connector and workload support plus Query History search.


### [Open House observability announcements: MCP server, AI Notebooks, and ClickStack Cloud](https://yomu.fyi/post/open-house-observability-announcements-mcp-server-ai-notebooks-and-cli.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 27, 2026
- Type: Announcement

At Open House, ClickHouse announced three observability updates: ClickStack Cloud entered private preview, AI Notebooks entered beta for Managed ClickStack, and a ClickStack MCP server was introduced in open-source ClickStack. ClickStack Cloud is a fully managed, serverless platform that accepts OpenTelemetry data through a managed endpoint, while Managed ClickStack is generally available for teams controlling collectors, ingestion, compute sizing, schema design, and tuning. AI Notebooks provide a persistent, shareable investigation workspace in which prompts, queries, charts, reasoning steps, findings, branches, intermediate results, and generated queries remain visible and editable. The MCP server exposes the investigative tools used by Notebooks to external agents, routing requests through optimized searches, aggregations, and visualizations rather than relying entirely on ad hoc SQL generation; it can also persist findings in dashboards. Together, the announcements present a choice between managed convenience, operational control, and programmable observability workflows.


### [Introducing ClickStack Cloud: Serverless observability powered by ClickHouse](https://yomu.fyi/post/introducing-clickstack-cloud-serverless-observability-powered-by-click.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 27, 2026
- Type: Announcement

ClickStack Cloud is entering private preview as a fully managed, serverless observability service built on ClickHouse for teams that want ClickHouse-powered logs, metrics, and traces without operating the underlying infrastructure. Users send OpenTelemetry data to a managed OTLP endpoint, while the service handles ingestion, durable buffering through an object-storage-backed event queue, scaling, storage, and query serving, then exposes telemetry in the ClickStack UI with integrated authentication and RBAC. The preview separates ingestion and query resources and is developing automatic tuning based on query patterns, including materialized fields, primary-key adjustments, materialized views, and indexes. Dedicated query compute for agentic and high-volume analytical workloads is planned after the preview, while pricing remains unfinished and the existing Managed ClickStack offering remains intended for teams requiring direct infrastructure and schema control.


### [You don’t need to pick one: how Sentry and OpenTelemetry work together](https://yomu.fyi/post/you-don-t-need-to-pick-one-how-sentry-and-opentelemetry-work-together.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Lazar Nikolov
- Published: May 27, 2026
- Type: Explainer

Integrating Sentry into systems already instrumented with OpenTelemetry does not require replacing existing backend SDKs or rewriting service instrumentation. A hybrid architecture pairs the Sentry SDK on the frontend for browser tracing, Session Replay, and logs with existing OpenTelemetry configurations across backend services. Frontend requests propagate W3C traceparent headers to backend endpoints, which then export traces and logs over OTLP directly to Sentry endpoints or through an intermediate OpenTelemetry Collector. Ingested telemetry attaches backend spans and standard Python logs to the initial frontend user actions, establishing a unified distributed trace across the entire request path. Although Sentry's OTLP ingest currently supports logs and traces rather than metrics, dedicated backend Sentry SDKs remain an optional addition later for backend exception tracking and profiling.


### [The product analytics you already have](https://yomu.fyi/post/the-product-analytics-you-already-have.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Rahul Chhabria
- Published: May 21, 2026
- Type: Explainer

Software engineers frequently rely on separate product analytics tools and teams to answer user behavior questions, creating disjointed data models and delayed insights. Existing debugging telemetry within tools like Sentry—specifically spans, structured logs, and application metrics—can serve as the primary source for product metrics. Developers enrich request spans with business-level attributes, record discrete business events through wide structured logs, and track KPIs with metric counters and distributions. This unified telemetry links business outcomes directly to underlying traces, errors, releases, and source code lines, enabling immediate root-cause analysis and automated alerts. While multi-session cohort and funnel aggregation remains a current tooling gap, standard telemetry solves day-to-day adoption, performance, and revenue questions.


### [Automate Data & KPI Monitoring with SQL Alerts](https://yomu.fyi/post/automate-data-kpi-monitoring-with-sql-alerts.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Srilekha Dornadula
- Published: May 19, 2026
- Type: Announcement

Databricks SQL Alerts is generally available as an automated alternative to manually checking dashboards and rerunning monitoring queries for business KPIs, pipeline health, and data quality. Teams define a SQL query, evaluation condition, schedule, and notification destinations, then receive alerts when results cross configured guardrails; standalone alerts can also run within Lakeflow Jobs. GA supports authoring in the SQL editor, notifications through email, Slack, PagerDuty, Microsoft Teams, or webhooks, and production management through Git, Declarative Automation Bundles, APIs, Terraform, and SDKs. The post’s revenue example compares yesterday’s result with a seven-day average and triggers when revenue\_pct\_change is below -5%, while the SQL Alert task can expose OK, TRIGGERED, or ERROR to downstream pipeline logic.


### [What's new in ClickStack - April 2026](https://yomu.fyi/post/what-s-new-in-clickstack-april-2026.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: The ClickStack Team
- Published: May 19, 2026
- Type: Announcement

ClickStack's April release introduces several core enhancements across querying, alerting, and dashboard visualization capabilities. The platform added SQL-powered alerting, allowing teams to express complex statistical rules like rolling baselines and anomaly detection using native ClickHouse SQL and window functions without external pipelines. To improve search and filtering performance across common observability workloads, the default otel\_logs schema was redesigned with ClickHouse text indexes and adjusted ordering keys. Furthermore, heatmaps transitioned into a first-class chart type inside the shared visualization system for latency distribution tracking. The update also delivers persistent alert execution error reporting, expanded threshold comparison operators, and per-series number formatting across dashboards.


### [How the D. E. Shaw group powers high-cardinality observability at scale with ClickHouse](https://yomu.fyi/post/how-the-d-e-shaw-group-powers-high-cardinality-observability-at-scale.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: ClickHouse
- Published: May 15, 2026
- Type: Explainer

Inside D. E. Shaw group, a global investment and technology development firm, millions of compute workloads run monthly across its high-performance computing infrastructure. To support capacity planning and resource allocation, it collects task-level metrics for requested and utilized compute, producing millions of distinct time series that its previous observability platform could not accommodate. Engineers benchmarked databases on identical hardware and datasets; ClickHouse ingested approximately 3.5 million samples per second on one node versus roughly 480,000 for a competing solution, while similar queries completed in seconds instead of timing out. They retained InfluxDB line protocol, batched messages into ClickHouse, and built disk-based tooling to transform and stream historical data; the resulting four-server NVMe cluster stores about 68 TB compressed and sustains 530k records per second on average. Materialized views support capacity-planning dashboards, while OpenTelemetry tracing extends the platform to other observability workloads, with SQL migration and data modeling requiring engineering effort.


### [ClickHouse vs Prometheus for High Cardinality, Part 2: Cardinality in ClickHouse](https://yomu.fyi/post/clickhouse-vs-prometheus-for-high-cardinality-part-2-cardinality-in-cl.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Rory Crispin, Dale McDiarmid
- Published: May 15, 2026
- Type: Explainer

Traditional time-series databases like Prometheus experience write amplification, memory overhead, and operational challenges when high-cardinality label combinations churn rapidly. ClickHouse alters these economics by abandoning dedicated series objects in favor of a wide events model, storing telemetry as rows in columnar tables with dynamic Map attributes and metric measurements. Dynamic attributes can be handled using sharded Map types alongside materialized columns and text indexes for frequently queried labels. This design trades ingestion overhead for query-time processing, relying on columnar compression, partition pruning, and parallel scans to compute aggregations on demand. ClickHouse avoids the ingest-side bottlenecks of high cardinality, though Prometheus remains preferable for moderate cardinality, native PromQL semantics, and single-series lookups.


### [ClickHouse vs Prometheus for High Cardinality, Part 1: Understanding the Problem](https://yomu.fyi/post/clickhouse-vs-prometheus-for-high-cardinality-part-1-understanding-the.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Rory Crispin, Dale McDiarmid
- Published: May 14, 2026
- Type: Explainer

Observability systems frequently encounter high cardinality when metrics contain numerous unique label combinations across multidimensional cloud infrastructure. In Prometheus, every unique label combination instantiates an independent time series that incurs metadata overhead, requires in-memory memSeries registration, and continually expands inverted index posting lists. The ingestion engine stores samples in two-hour Head blocks using compressed 120-sample chunks with XOR-based encodings, but rapid series creation and elevated scrape frequencies multiply chunk allocations and memory consumption. When executing read operations, broad aggregations and regular expressions require intersecting expansive posting lists and decoding entire chunks sequentially because the engine cannot push value predicates directly into compressed storage blocks. Consequently, managing high-cardinality workloads in series-oriented architectures forces operational trade-offs between system stability and granular visibility into ephemeral components.


### [New ways to agentically build and edit dashboards](https://yomu.fyi/post/new-ways-to-agentically-build-and-edit-dashboards.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Ben Coe, Steve Zegalia
- Published: May 14, 2026
- Type: Announcement

Pre-configured monitoring templates often fail to meet organization-specific needs, forcing developers to manually build dashboards widget by widget. Sentry introduced AI-agent-powered dashboard creation and editing in open beta, alongside CLI management tooling and template revisions. Users can prompt an AI chat interface to build or adjust dashboards, duplicate and customize uneditable Sentry-built templates, or execute dashboard commands directly in their terminal. Dashboards leverage issue, tracing, and application metric data across frontend, backend, mobile, and framework-specific views like Next.js and Laravel. Edits made through the UI, CLI, or AI agent are automatically tracked in a revision history that supports one-click rollbacks.


### [ClickStack SQL Charting and Alerting](https://yomu.fyi/post/clickstack-sql-charting-and-alerting.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Drew Davis, Dale McDiarmid
- Published: May 13, 2026
- Type: Announcement

ClickStack introduced SQL-based visualizations and SQL-based alerting to allow users to build charts and alerts using arbitrary ClickHouse SQL queries. Traditional query builders trade analytical flexibility for simplicity, preventing teams from easily expressing multi-stage computations like rolling baselines, anomaly detection, or dynamic SLO calculations. To maintain interactivity within dashboards, the system supports ClickHouse query parameters and macros that inject active time ranges, interval sizes, dashboard filters, and source tables. Furthermore, SQL-based alerting evaluates the numeric columns output by these queries, such as returning binary flags when values exceed statistical boundaries. By shifting computational logic directly into SQL, teams can implement complex operational checks without relying on static external thresholds.


### [From vibe code to production-ready: observability for Next.js and Supabase apps](https://yomu.fyi/post/from-vibe-code-to-production-ready-observability-for-next-js-and-supab.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Sergiy Dybskiy
- Published: May 11, 2026
- Type: Problem & solution

AI-assisted software development often generates functional code that lacks proper instrumentation, leading to unindexed database queries, N+1 patterns, and fragmented connection strategies. While Supabase provides built-in query analytics, security advisors, and subsystem logs, it cannot trace distributed requests across the entire application stack. Integrating Sentry bridges this visibility gap by establishing distributed traces across Next.js frontends, Deno-based Supabase Edge Functions, and Postgres databases. Teams can forward Supabase logs into dedicated Sentry projects and configure runtime integrations to automatically detect performance bottlenecks such as slow spans and Web Vitals regressions. Furthermore, integrating Model Context Protocol servers allows Sentry's Seer debugger to identify root causes and generate automated fixes for emerging issues.


[Newer posts](https://yomu.fyi/topic/monitoring/page/4.md) · [Older posts](https://yomu.fyi/topic/monitoring/page/6.md)