---
title: "Building high-performance full-text search for object storage"
description: "ClickHouse redesigned its native full-text index to address latency bottlenecks caused by random read patterns on remote object storage. Instead of using a Finite State Transducer dictionary, the new architecture organizes indexed tokens into fixed-size sorted blocks compressed with front-coding. Each data part separates index data across a small in-memory sparse index file, a sequential dictionary file, and a posting list file. This layout allows the query engine to pinpoint token locations with minimal random I/O and resolve multiple full-text functions directly from index files. Consequently, ClickHouse Cloud maintains predictable search latency across shared object storage while matching local disk performance and scaling through parallel query execution."
---

# Building high-performance full-text search for object storage

[Clickhouse](https://yomu.fyi/company/clickhouse) · Elmi Ahmadov, Jimmy Aguilar, and George Larionov · Mar 24, 2026

**Type:** Problem & solution

## Summary

ClickHouse redesigned its native full-text index to address latency bottlenecks caused by random read patterns on remote object storage. Instead of using a Finite State Transducer dictionary, the new architecture organizes indexed tokens into fixed-size sorted blocks compressed with front-coding. Each data part separates index data across a small in-memory sparse index file, a sequential dictionary file, and a posting list file. This layout allows the query engine to pinpoint token locations with minimal random I/O and resolve multiple full-text functions directly from index files. Consequently, ClickHouse Cloud maintains predictable search latency across shared object storage while matching local disk performance and scaling through parallel query execution.

## Context

Remote object storage in ClickHouse Cloud provides high throughput but significantly higher access latency than local disks. The previous text index used a Finite State Transducer dictionary structure that relied on small, scattered random reads, causing end-to-end query latency bottlenecks when reading from remote object storage.

## Approach / What changed

ClickHouse redesigned the text index into three separate files per data part: an in-memory sparse index, a dictionary file, and a posting list file. The dictionary groups sorted tokens into fixed-size blocks compressed with front-coding, referencing offsets in posting lists. This block-based layout prioritizes sequential access, allowing lookups to navigate via the sparse index and answer queries without reading the raw indexed text column.

## Takeaways

- The text index splits data across three separate files per part: the dictionary file (text\_idx.dct.idx), the sparse dictionary index file (text\_idx.idx), and the posting list file (text\_idx.pst.idx).
- Dictionary tokens are grouped into fixed-size blocks of 512 entries by default and compressed using front-coding to replace full strings with prefix lengths and suffixes.
- Negation functions such as !=, NOT IN, and NOT LIKE are unsupported because the text index only tracks terms and posting lists that exist in the table.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Performance](https://yomu.fyi/topic/performance), [Search](https://yomu.fyi/topic/search)

- Source: [Clickhouse](https://clickhouse.com/blog/clickhouse-full-text-search-object-storage)
- Source URL: https://clickhouse.com/blog/clickhouse-full-text-search-object-storage
- Ingested by Yomu: 2026-08-28T01:22:55.926Z

[Read original post](https://clickhouse.com/blog/clickhouse-full-text-search-object-storage)