ClickhouseProblem & solution
iSAM Funds, a UK-based systematic asset manager, built an options research platform on ClickHouse Cloud to handle the 10,000x data fan-out from underlying instruments to related options. Migrating from Postgres, the team achieved 35-50x compression and increased single-core ingestion from 8,000 to 800,000 rows per second, while ClickHouse’s native Apache Arrow support eliminated encoding reshuffling. ClickHouse’s columnar storage and vectorized execution made queries fast enough to run on demand, allowing the team to drop pre-computed rollups and materialized views and remove a layer of pipeline complexity. The managed service integrates with iSAM’s AWS footprint and enterprise authentication, pushes backups to iSAM’s own buckets for developer use, and lets them scale instances up for large backfills then down to control costs. The platform gives researchers fast, interactive access to full options history, shortening the time from hypothesis to tested strategy.
iSAM Funds needed to store and query full options history for research, but the 10,000x data fan-out from underlying instruments to related options created a massive data engineering challenge. Postgres could not handle the volume, and simple horizontal partitioning was ineffective due to uneven market activity across exchange products.
The team migrated from Postgres to ClickHouse Cloud, using its columnar storage and vectorized execution to achieve 35-50x compression and 100x faster single-core ingestion. They integrated with their existing AWS infrastructure and enterprise authentication, and used ClickHouse’s native Arrow support to avoid encoding reshuffling. The managed service allowed them to scale instances up for large backfills and down afterward, avoiding peak-sizing on-premises hardware.
ClickHouse’s columnar storage achieved 35-50x compression over Postgres for time-series options data without special tuning.
Single-core ingestion jumped from 8,000 to 800,000 rows per second, and ClickHouse’s performance eliminated the need for pre-computed rollups and materialized views.
The team uses ClickHouse Cloud’s ability to scale instances up for large backfills and down afterward, avoiding the cost of peak-sizing on-premises hardware.
ClickHouse's columnar storage and vectorized execution made queries fast enough to run on demand, so the team could store full history without pre-aggregation. The 35-50x compression also reduced storage footprint significantly.
iSAM required integration with its existing AWS footprint and enterprise authentication stack. ClickHouse Cloud provided direct connectivity over AWS and supported pushing backups to iSAM's own buckets for developer use.
They scale the instance up for large backfills and let it scale back down afterward, rather than sizing an on-premises machine for peak load. Steve Barham calls this a cost-efficient way of renting expensive hardware.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
BYOK lets a system run on assets you supply — your model credential or your private knowledge — instead of the vendor's defaults. This walkthrough traces six steps from registration to an audited response.
A five-checkpoint walkthrough of BYOC: name the control you need, split control and data planes, build the landing zone, govern data in place, then operate inside your own tenant.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
