# ObsessionBench

> ObsessionDB vs ClickHouse Cloud on ClickBench: latency, consistency and cost, measured in the open.

A public benchmark of ObsessionDB (https://obsessiondb.com) against ClickHouse Cloud (https://clickhouse.com/cloud).
Primary metric: Typical query time (geometric mean) (`geomean_query_ms`, ms, lower is better).
Built by Letter (The Letter Company) for ObsessionDB, a Letter customer. The code, data source and raw results are in the repo.

- Leaderboard: https://obsessionbench.com/
- Methodology: https://obsessionbench.com/methodology
- Replicate: https://obsessionbench.com/replicate
- All runs: https://obsessionbench.com/results
- Raw latest results: https://obsessionbench.com/results/latest.json

## Latest run

Ran at: 2026-10-31T22:47:51Z

Summary: Typical query time (geometric mean) (server-side, lower is better) on the 43 ClickBench queries over 10,000,000 rows, 5 warm runs each: ObsessionDB 64.1 ms, ClickHouse Cloud 85 ms.

Leaderboard:

1. ObsessionDB (subject) [ok]: geomean_query_ms=64.1 ms, suite_seconds=10 s, p50_latency_ms=71 ms, p99_latency_ms=2,777 ms, concurrent_p99_ms=11,499 ms, load_rows_per_sec=55,388 rows/s, cost_per_1k_queries_usd=$0.029454
2. ClickHouse Cloud (clickhouse-cloud) [ok]: geomean_query_ms=85 ms, suite_seconds=11.3 s, p50_latency_ms=93 ms, p99_latency_ms=4,643 ms, concurrent_p99_ms=15,635 ms, load_rows_per_sec=337,843 rows/s, cost_per_1k_queries_usd=$0.029847

## Replicate

# Replicate ObsessionBench

For a person or an AI agent: how to reproduce the published results and check them.

## 1. What you need

Node.js 24, npm and git, and network access to each contender's service. A contender that needs
credentials reads them from the environment variables named in `.env.example`; without them, the
contender is recorded as `skipped`, never scored.

- ObsessionDB (`subject`): `OBSESSIONDB_HOST`, `OBSESSIONDB_PORT`, `OBSESSIONDB_USER`, `OBSESSIONDB_PASSWORD`
- ClickHouse Cloud (`clickhouse-cloud`): `CLICKHOUSE_CLOUD_HOST`, `CLICKHOUSE_CLOUD_PORT`, `CLICKHOUSE_CLOUD_USER`, `CLICKHOUSE_CLOUD_PASSWORD`

Use your own account on each service. `BENCH_SCALE` sets the dataset's size and `BENCH_REPETITIONS`
how many times each query is timed; the defaults are what the published results used.

## 2. Get the code

```sh
git clone <this repo>
cd obsessiondb-bench
npm install
```

## 3. Run it

```sh
npm run bench
```

It runs every contender in `bench.json` on every scenario and writes `results/latest.json` and
`results/history/<YYYY-MM-DD>.json` (the shape is in `results/README.md`).

## 4. Compare with the published results

The published file is `results/latest.json` on `main`.
The same results, readable: https://obsessionbench.com

Compare each contender's metrics. Run-to-run noise is expected; METHODOLOGY.md says what counts
as a real difference. The primary metric is `geomean_query_ms` (lower is better).

## 5. Report a difference

Open an issue on the repo with your `results/latest.json` and the
`environment` it recorded.