HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
ClickHouse
ClickBench
Shell2022–2026
ClickHouse / ClickBench

ClickBench

ClickBench: a Benchmark For Analytical Databases

Data & DatabasesTesting & QualityDevOps & Cloud
Explore on GitHubRead Hacker News discussions
33Distinct people
73Mentions
1Explicit recommenders
1.2KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

ClickBench: a Benchmark For Analytical Databases

GitHub

Where it is used

  • rtabench — RTABench uses the Clickbench framework for benchmarking, but it introduces a new dataset and query set that better represents real-time analytics inside applications. All tools, datasets, and benchmark results are available on GitHub, where we welcome contributions for expanding RTABench to support additional databases and optimizations. Source

What people are saying

Also I recommend to read the methodology here: https://github.com/ClickHouse/ClickBench/blob/main/README.md Making an open-source, easily reproducible benchmark with clear…

zX41ZdbW used a recommendation phrase. 2 indexed comments match. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:03:16 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year73 dated mentions

Observed mentions: 2022-07-13 16:03:09 UTC to 2026-11-02 01:46:48 UTC.

01020302022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 10 mentions; 1 submissions, 9 comments20222023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 20 mentions; 0 submissions, 20 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 8 mentions; 1 submissions, 7 comments20242025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 8 mentions; 1 submissions, 7 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 27 mentions; 1 submissions, 26 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC1019
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC20020
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC817
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC817
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC27126

What Hacker News says

73 sources
malisper·Nov 2, 2026
I work on pgrust. We shouldn't have submitted the preview when we did: it was still in active development, and its SQL coverage was far too narrow, though there was nothing in it giving it an unfair performance advantage For the other two claims in the removal PR, we wrote scripts you can use to verify the behavior of the original 0.4-preview binary: - Per-query kernels did not serve the benchmark queries[0] - The preview's fast repeat LIKE queries came from a dictionary filter cache, not a cached query result[1] Our open-sourced 0.4-beta has now been merged and is back on ClickBench[2][3] Across all queries and instance sizes on ClickBench, 0.4-beta is ~7% faster overall than the 0.4-preview binary that was removed. If 0.4-preview's numbers had come from hardcoded kernels or faked results, the open-source general engine wouldn't be
Read the rest
beating it [0] https://pgrust.com/downloads/archive/2026-09-clickbench-snap... [1] https://pgrust.com/downloads/archive/2026-09-clickbench-snap... [2] https://benchmark.clickhouse.com/ [3] https://github.com/ClickHouse/ClickBench/pull/2751
In Postgres 19 is late because so many more bugs are being found now
malisper·Oct 25, 2026
Update: ClickHouse finished the preliminary benchmarks for 0.4-beta[0], the version we've open-sourced. If you compute the geomean across all queries and instance sizes, it comes out as 7% faster than the 0.4-preview entry[1] that was accused of cheating. To give you a sense of how fast it is, pgrust on a c6a.2xl is faster than ClickHouse running on a c6a.metal, a machine with 24x as many cores and costs 24x as much. Edit: And now it's been accepted: https://benchmark.clickhouse.com/ [0] https://github.com/ClickHouse/ClickBench/commit/56535d29e825... [1] https://github.com/ClickHouse/ClickBench/commit/89cc76e1e4e2...
In We should be more ambitious with the software we build
0conclusive-dem·Oct 25, 2026
this feels reasonable. for communities like r/selfhosted there has been an influx of new projects, many of which initially get dismissed. this more generous attitude can likely go a long way. unfortunately though pgrust appears to be at best misleading, at worst vibeslop: https://github.com/ClickHouse/ClickBench/pull/2165 hacknews thread isn’t much prettier either https://news.ycombinator.com/item?id=50205816
In We should be more ambitious with the software we build
hitekker·Oct 25, 2026
For those curious, see the accusation here https://github.com/ClickHouse/ClickBench/pull/2165 and then the HN thread https://news.ycombinator.com/item?id=50205816 HN is waiting for the source code for pgrust 0.4-preview to be released, the version which allegedly cheated Clickhouse benchmark. So far, the people (person?) behind pgrust has not committed to releasing the offending code, and has instead offered other code (0.4-beta, not preview).
In We should be more ambitious with the software we build
malisper·Oct 23, 2026
Hi! I'm the creator of pgrust. Let me explain what happened. For the most part, the actual accusations are false. This submission was an earlier version of pgrust 0.4 called pgrust 0.4-preview. We just open-sourced a newer version called pgrust 0.4-beta yesterday, and you can fully audit it here[0]. Rather than contest the accusations on GitHub, we figured it would be easier to complete pgrust 0.4 and open-source it to show that we're being honest. You can also verify what I'm saying yourself against the 0.4-preview binary we submitted to ClickBench here[1]. I don't have a way to prove this is the exact binary submitted, but if ClickHouse provided the SHA of the binary they tested against, it would match. The three main accusations boil down to: 1) We used handwritten code optimized against each query in the benchmark, 2) We cached the re
Read the rest
sults between query runs, and 3) We didn't support queries outside of the 43 ClickBench queries. First, we did not use handwritten queries. The code Alexey pointed out does not run during the benchmark. The design of 0.4 relies on a JIT compiler that compiles your SQL to a Lisp-based intermediate representation that we call Gravity IR that we then convert to x86 and ARM assembly. While developing 0.4, I wrote handwritten code against the IR for benchmarking purposes. The idea was that if queries written in our IR are fast, and we can compile good enough IR fast, then our new query engine will be fast. This is also the reason the directory those "kernels" are in is called "irprobe" - it was a probe of our IR to make sure it was fast. Our 0.4-preview submission went through the pipeline: SQL query -> Gravity IR -> assembly. Second, the "result-cache" Alexey observed is the result of a cache we use to cache which predicates match which strings in our dictionaries we use for dictionary compression. Alexey wasn't aware of this cache and assumed we were just caching the query results. On ClickBench you are allowed to use caches like this that hold intermediate data[2]. ClickHouse has its own cache similar to this called the Condition Cache[3]. In our newest submission, we were upfront about all the caching we do and how to disable each cache[4]. The third claim that we didn't support other queries outside of the 43 ClickBench queries is partially false, but does have some ground to it. As I said before, pgrust 0.4-preview was an early version of pgrust 0.4. Once our new JIT compiler supported enough SQL to handle the ClickBench queries, we decided to submit it to ClickBench. AFAIK, ClickBench has no rules about how much SQL you need to support. On top of that, we did ship that we did ship 0.4-preview with our old query engine. It would have been a two-line change to fall back to it and support a much broader set of queries outside what our new engine could support. Those queries would still be fast, but not quite as fast as our new engine was. Our new 0.4-beta release does now support all Postgres SQL (minus some bugs around CTEs and MERGE) To sum up, pgrust 0.4-preview was an early build of pgrust 0.4 while it was in development. It was a real query engine and actually executed queries. I acknowledge that we submitted it prematurely, when it supported a narrow set of SQL, and we should have ensured broader SQL coverage before submitting it. With our new submission that solves these problems, and we're hoping for pgrust to be reinstated. If there's anything off about the submission, please point it out and we're happy to fix it. [0] https://github.com/malisper/pgrust/releases/tag/v0.4-beta [1] https://pgrust.com/downloads/archive/2026-09-clickbench-snap... [2] https://github.com/ClickHouse/ClickBench/blob/main/README.md [3] https://clickhouse.com/docs/concepts/features/performance/ca... [4] https://github.com/ClickHouse/ClickBench/pull/2751
In Remove pgrust: the entry is cheating (per-query kernels and result caching)
acalvino4·Oct 23, 2026
Even before the ai age, people published inaccurate/misleading benchmarks all the time. Sometimes those were actual cheating, and other times an honest mistake. What matters is that the author acknowledges the issue and attempts to correct the mistake, which clearly seems to have happened here: https://github.com/ClickHouse/ClickBench/pull/2751 Most of this thread doesn't seem to acknowledge that, and feels more like anti-ai psychosis using this incident as an anecdote to justify the pre-existing hatred of ai-generated projects.
In Remove pgrust: the entry is cheating (per-query kernels and result caching)

Showing the 15 most recent of 73 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.