ObsessionBench
ObsessionDB vs ClickHouse Cloud on ClickBench: latency, consistency and cost, measured in the open.
Leaderboard
p99 query latency (server-side, lower is better) on the 43 ClickBench queries over 10,000,000 rows, 5 warm runs each: ObsessionDB 2,786 ms, ClickHouse Cloud 4,742 ms.
| # | Contender | Status | p99 query latencylower is better | Median query latencylower is better | Full ClickBench suite timelower is better | p99 latency with 8 concurrent clientslower is better | Data load speedhigher is better | Cost per 1,000 queries (list price)lower is better |
|---|---|---|---|---|---|---|---|---|
| 1 | ObsessionDB | ok | 2,786 ms | 98 ms | 11.4 s | — | 66,977 rows/s | $0.032009 |
| 2 | ClickHouse Cloud | ok | 4,742 ms | 92 ms | 11.6 s | 32,523 ms | 332,910 rows/s | $0.030234 |
- ObsessionDB: Server-side times unavailable (system.query_log had 1190 of 1235 queries after waiting 2 minutes), so the latency metrics are null; end-to-end times are in details. 13 of 1032 concurrent queries failed and are left out of concurrent_p99_ms.
End-to-end times at the client
Measured where the bench ran, so they include the network distance to each service. The leaderboard uses server time.
| Contender | p99 query latency | Median query latency | Full ClickBench suite time | p99 latency with 8 concurrent clients |
|---|---|---|---|---|
| ObsessionDB | 2,790 ms | 102 ms | 11.5 s | 23,034 ms |
| ClickHouse Cloud | 4,752 ms | 103 ms | 12 s | 32,533 ms |
Environment
- runner
- Linux 6.1.188-maritime1 x64, 2 x Intel(R) Xeon(R) Processor @ 3.20GHz, 8 GB RAM
- node
- v24.22.1
- harness_commit
- b6505d79eb569ae1b6324ee85555a06c186e3ca4
- clickbench
- https://github.com/ClickHouse/ClickBench @ 0cb36ab1a62c445414828aae1c9dccacb71af37b (clickhouse-cloud/create.sql, clickhouse-cloud/queries.sql)
- scale
- BENCH_SCALE=10: 10 of 100 ClickBench partitioned parquet files (https://datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_{0..9}.parquet)
- repetitions
- BENCH_REPETITIONS=5
- query_timeout
- 60 s
- concurrency
- 8 clients x 3 rounds, seed 20261031
- query_settings
- use_query_cache=0, max_execution_time=60, log_queries=1
- client
- ClickHouse HTTP interface over HTTPS, node:https with keep-alive, wait_end_of_query=1
- subject_version
- 26.8.2.1
- subject_region
- eu-central (as set up for the run; see METHODOLOGY.md)
- subject_instance
- Free developer tier, single node, 4 vCPU / 8 GB RAM
- subject_price
- $0.438356/hour, Nearest paid ObsessionDB equivalent of the free 4 vCPU / 8 GB instance: one 16 GB node (2 compute units, 4 vCPU), the smallest paid size with at least 4 vCPU and 8 GB, at the listed $20 per GB of memory per node per month = $320/month, divided by 730 hours a month (the free tier is not used as the price), read 2026-10-31 from https://obsessiondb.com/pricing
- clickhouse-cloud_version
- 26.6.1.2455
- clickhouse-cloud_region
- AWS eu-central-1 (as set up for the run; see METHODOLOGY.md)
- clickhouse-cloud_instance
- 1 replica, fixed 12 GiB / 3 vCPU (autoscaling off: min = max)
- clickhouse-cloud_price
- $0.39375/hour, ClickHouse Cloud Basic plan (its 1-replica 12 GiB / 3 vCPU size) in AWS eu-central-1: $0.2625 per compute-unit-hour (1 unit = 8 GiB / 2 vCPU) x 1.5 units, read 2026-10-31 from https://clickhouse.com/pricing
What is measured
- Data load. Create the standard ClickBench hits table and load the same 10 million rows (the first 10 partitioned ClickBench parquet files) into each service. The load runs server-side from the public dataset URLs, so the sandbox never holds the data. Timed from first insert to the final row count matching.
- ClickBench, warm. All 43 ClickBench queries, one client. Each query runs once untimed as a warm-up, then 5 timed runs. Per-query p50 and p99 come from the timed runs; suite time is the sum of per-query medians.
- Concurrent load. 8 clients each run the 43 queries in a shuffled order (same seed for every contender) for 3 rounds after the warm-up. Reports p99 latency across all queries, to show consistency under load.
How it is run and why it is fair: methodology. Run it yourself: replicate. Every run: results.