JAjabart·Nov 6, 2023
Looks like a recent PR bumped benchmark.clickhouse.com to DuckDB v0.9 on the 3rd.
https://github.com/ClickHouse/ClickBench/pull/141
In Updates to the H2O.ai db-benchmarkClickHouse
ClickBench
Shell2022–2026
ClickHouse / ClickBench
ClickBench
ClickBench: a Benchmark For Analytical Databases
33Distinct people
73Mentions
1Explicit recommenders
1.2KGitHub stars
What to know before exploring
What it is
ClickBench: a Benchmark For Analytical Databases
GitHub
Where it is used
- rtabench — RTABench uses the Clickbench framework for benchmarking, but it introduces a new dataset and query set that better represents real-time analytics inside applications. All tools, datasets, and benchmark results are available on GitHub, where we welcome contributions for expanding RTABench to support additional databases and optimizations. Source
What people are saying
Also I recommend to read the methodology here: https://github.com/ClickHouse/ClickBench/blob/main/README.md Making an open-source, easily reproducible benchmark with clear…
zX41ZdbW used a recommendation phrase. 2 indexed comments match. This is a phrase match, not a verdict.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 8:03:49 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year73 dated mentions
Observed mentions: 2022-07-13 16:03:09 UTC to 2026-11-02 01:46:48 UTC.
* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 10 | 1 | 9 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 20 | 0 | 20 |
| 2024 | 2024-01-01 00:00:00 UTC | 2025-01-01 00:00:00 UTC | 8 | 1 | 7 |
| 2025 | 2025-01-01 00:00:00 UTC | 2026-01-01 00:00:00 UTC | 8 | 1 | 7 |
| 2026 (partial) | 2026-01-01 00:00:00 UTC | 2027-01-01 00:00:00 UTC | 27 | 1 | 26 |
What Hacker News says
20 sourcesIt's hardly comparable with ClickHouse. Even loading a table with 100M rows is not an easy endeavor in Pinot: https://github.com/ClickHouse/ClickBench/blob/main/pinot/ben...
In Apache Pinot 1.0I tried Boiling Data, but the product looks dead or abandoned: https://github.com/ClickHouse/ClickBench/issues/125 It does not work at all.
In A LLM+OLAP SolutionAlso I recommend to read the methodology here: https://github.com/ClickHouse/ClickBench/blob/main/README.md
Making an open-source, easily reproducible benchmark with clear rules allowed having more than 30 different DBMS in a single benchmark. Many of the results were submitted or improved by the vendors of the corresponding systems.
There is a comparison section that lists limitations and drawbacks in comparison with other benchmarks.
In Common Pitfalls in Database Performance Testing (2018) [pdf]Yes, setting between Hydra and Postgres (tuned) are somehow aligned. But what Postgres (tuned) also creates are bunch of indexes (https://github.com/ClickHouse/ClickBench/blob/main/postgresq...) which adds around ~50GB of extra storage.
In Hydra: Column-Oriented Postgreshttps://github.com/ClickHouse/ClickBench/blob/main/postgresq...
basically larger buffers and more concurrent workers.
In Hydra: Column-Oriented Postgres> 0 - https://github.com/ClickHouse/ClickBench/blob/main/postgresq...
that postgres config is very underpowered, it has only 8 workers per gather while machine has 192 vcpus.
In Show HN: Hydra 1.0 – open-source column-oriented PostgresSince benchmarks can be misleading I want to point out that the differences between Hydra and the "tuned"[0] PostgreSQL (which are some very basic settings) are a lot less convincing, with plain old PG coming ahead on quite a few: https://tinyurl.com/eju9tht2
I also noticed quite a bit of parity between Hydra and Citus on data set size. Is Hydra a fork of Citus columnar storage?
0 - https://github.com/ClickHouse/ClickBench/blob/main/postgresq...
In Show HN: Hydra 1.0 – open-source column-oriented PostgresHere are examples of MongoDB aggregations: https://github.com/ClickHouse/ClickBench/blob/main/mongodb/q...
They are painful to write compared to SQL queries.
Although the commercial version of MongoDB has support for SQL, it's not available for general MongoDB users.
In PRQL: Pipelined Relational Query LanguageI've recently implemented a project to test and compare analytical databases, including cloud services: https://github.com/ClickHouse/ClickBench/
It compares Redshift and Athena as well.
In Amazon Redshift vs. Athena: A Performance ComparisonIt can be applied for a larger dataset, see https://github.com/ClickHouse/ClickBench/tree/main/clickhous... (100 billion records), but so far only ClickHouse has been tested on this volume - it takes too much time and cost to load this data into every DBMS.
In Can you even trust benchmarks these days? ClickHouse vs. Druid vs. RocksetFull disclosure: I work at StarTree, which is powered by Apache Pinot.
ClickHouse's ClickBench is a good general tool. However, it's not the end-all, be-all of performance benchmarking and testing. Its results may or may not be applicable for guidance on the performance of your specific use case when you get to production.
It is definitely a stab at getting an objective suite of tools for the real-time analytics space. But just like you had YCSB as a good general performance test, eventually a subset of users wanted something specific for Cassandra and Cassandra-like databases (DSE, ScyllaDB, etc.), so you eventually saw cassandra-stress. We have to consider cases where certain databases may need to have testing suites that really capture their capabilities.
ClickHouse themselves publishes a list of Limitations that everyone should keep i
Read the rest
n mind as they run ClickBench:
https://github.com/ClickHouse/ClickBench/#limitations
CelerData (based on StarRocks) also wrote up this:
https://celerdata.com/blog/what-you-should-know-before-using...
Plus, I want to direct people to the discussion generated when ClickBench was first posted to HN:
https://news.ycombinator.com/item?id=32084571
As user AdamProut commented back at the time:
> It looks like the queries are all single table queries with group-bys and aggregates over a reasonably small data set (10s of GB)?
>I'm sure some real workloads look like this, but I don't think it's a very good test case to show the strengths/weaknesses of an analytical databases query processor or query optimizer (no joins, unions, window functions, complex query shapes ?).
> For example, if there were any queries with some complex joins Clickhouse would likely not do very well right now given its immature query optimizer (Clickhouse blogs always recommend denormalizing data into tables with many columns to avoid joins).
So, again, ClickBench is a good (great) beginning. As an industry we should not let it be seen as the end. I'd be interested in the community's opinions on what and how we should be doing better.
I am the author of ClickBench: https://github.com/ClickHouse/ClickBench/
You can check the methodology described in the README and check the pull requests submitted by many database vendors.
In Can you even trust benchmarks these days? ClickHouse vs. Druid vs. RocksetOne way to add some trust is to make benchmarks open-source and reproducible:
https://github.com/ClickHouse/ClickBench/
(I work at ClickHouse)
In Can you even trust benchmarks these days? ClickHouse vs. Druid vs. RocksetI recommend checking ClickBench: https://github.com/ClickHouse/ClickBench
Disclaimer: I'm the author of ClickBench. It contains the comparison of Datafusion, DuckDB as well, and 30+ more DB engines in an open benchmark. The results of Datafusion were contributed by their team, and the results of DuckDB were contributed by their respective team.
In Comparison of DataFusion and DuckDB using TPC-H benchmarksShowing the 15 most recent of 20 sources. Enable JavaScript to search and browse every indexed mention.