GCP Pub/Sub connector for ClickPipes is now in Private Preview

Summary

ClickHouse has announced a GCP Pub/Sub connector for ClickPipes, now in Private Preview, to stream GCP infrastructure logs, application events, and operational signals into ClickHouse Cloud in real time. Previously, routing data from Pub/Sub into ClickHouse required GCS or a Dataflow pipeline; the connector instead manages subscription lifecycle, schema inference, and message delivery after a topic and target table are configured. It supports JSON, Avro, and Protobuf, Pub/Sub Schema Registry integration, automatic decompression, attribute-based filtering, and start positions of latest, earliest, or a specific timestamp. ClickPipes supports horizontal and vertical scaling, configurable replicas, automatic retries, built-in metrics and logs, and programmatic management through OpenAPI and the ClickHouse Terraform Provider. The connector is available upon request and free during Private Preview, with access offered through a waitlist or a ClickHouse Cloud account manager.

Takeaways

1. 1

   The connector supports JSON, Avro, and Protobuf, with native Pub/Sub Schema Registry integration and automatic mapping to ClickHouse native data types.

2. 2

   Pub/Sub seek options can start ingestion at the latest position, earliest position, or a specific timestamp; subscription filters can drop messages before they are pulled.

3. 3

   ClickPipes can be managed through OpenAPI and ClickHouse Terraform Provider 3.16.0+, and the Pub/Sub source is available free upon request during Private Preview.