HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
fivetran
benchmark
Shell2017–2019
fivetran / benchmark

benchmark

Benchmark data warehouses under Fivetran-like conditions

Data & DatabasesTesting & Quality
Explore on GitHubRead Hacker News discussions
2Distinct people
7Mentions
0Explicit recommenders
171GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Benchmark data warehouses under Fivetran-like conditions

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 2:04:58 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month7 dated mentions

Observed mentions: 2017-10-09 15:56:25 UTC to 2019-05-14 19:05:20 UTC.

01232017-10-01 00:00:00 UTC to 2017-11-01 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 commentsOct 20172017-11-01 00:00:00 UTC to 2017-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2017-12-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-01-01 00:00:00 UTC to 2018-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-02-01 00:00:00 UTC to 2018-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-03-01 00:00:00 UTC to 2018-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-04-01 00:00:00 UTC to 2018-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-05-01 00:00:00 UTC to 2018-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-06-01 00:00:00 UTC to 2018-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-07-01 00:00:00 UTC to 2018-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsJul 20182018-08-01 00:00:00 UTC to 2018-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-09-01 00:00:00 UTC to 2018-10-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2018-10-01 00:00:00 UTC to 2018-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-11-01 00:00:00 UTC to 2018-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-12-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-01-01 00:00:00 UTC to 2019-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-02-01 00:00:00 UTC to 2019-03-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2019-03-01 00:00:00 UTC to 2019-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-04-01 00:00:00 UTC to 2019-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-05-01 00:00:00 UTC to 2019-06-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsMay 2019

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Oct 20172017-10-01 00:00:00 UTC2017-11-01 00:00:00 UTC303
Nov 20172017-11-01 00:00:00 UTC2017-12-01 00:00:00 UTC000
Dec 20172017-12-01 00:00:00 UTC2018-01-01 00:00:00 UTC000
Jan 20182018-01-01 00:00:00 UTC2018-02-01 00:00:00 UTC000
Feb 20182018-02-01 00:00:00 UTC2018-03-01 00:00:00 UTC000
Mar 20182018-03-01 00:00:00 UTC2018-04-01 00:00:00 UTC000
Apr 20182018-04-01 00:00:00 UTC2018-05-01 00:00:00 UTC000
May 20182018-05-01 00:00:00 UTC2018-06-01 00:00:00 UTC000
Jun 20182018-06-01 00:00:00 UTC2018-07-01 00:00:00 UTC000
Jul 20182018-07-01 00:00:00 UTC2018-08-01 00:00:00 UTC000
Aug 20182018-08-01 00:00:00 UTC2018-09-01 00:00:00 UTC000
Sep 20182018-09-01 00:00:00 UTC2018-10-01 00:00:00 UTC202
Oct 20182018-10-01 00:00:00 UTC2018-11-01 00:00:00 UTC000
Nov 20182018-11-01 00:00:00 UTC2018-12-01 00:00:00 UTC000
Dec 20182018-12-01 00:00:00 UTC2019-01-01 00:00:00 UTC000
Jan 20192019-01-01 00:00:00 UTC2019-02-01 00:00:00 UTC000
Feb 20192019-02-01 00:00:00 UTC2019-03-01 00:00:00 UTC101
Mar 20192019-03-01 00:00:00 UTC2019-04-01 00:00:00 UTC000
Apr 20192019-04-01 00:00:00 UTC2019-05-01 00:00:00 UTC000
May 20192019-05-01 00:00:00 UTC2019-06-01 00:00:00 UTC101

What Hacker News says

7 sources
georgewfraser·May 14, 2019
As someone who's spent a lot of time working with TPC-DS [1] and talking to people about it [2], I see a couple areas that could be improved in this benchmark: 1. Total run time is not an appropriate way to summarize the performance across queries, because some queries take 100x longer than others. The appropriate way to summarize this kind of data is to use the geomean [3]. 2. The official TPC-DS queries make heavy use of grouping sets, which are a rarely-used SQL feature. I think TPC-DS is better if you rewrite the queries to eliminate grouping sets. 3. You used the exact same queries to "warm up" the data warehouse, and to test the performance. Some data warehouses (notably Redshift) aggressively cache intermediate compilation results, so they are much faster the second time they see a query or even a fragment of a query. To model a re
Read the rest
al user submitting queries interactively, you should use warmup queries that are similar to but not the same as the ones you use to measure performance. 4. You can solve the "vendor benchmarking their own product" problem by submitting a PR to our repo [4], which currently tests Redshift, Snowflake, BigQuery, Azure SQL DW, and Presto. We'd be happy to review it and endorse the timing if it meets our standards for fairness! [1] https://fivetran.com/blog/warehouse-benchmark [2] https://www.youtube.com/watch?v=XpaN-PqSczM [3] https://en.wikipedia.org/wiki/Geometric_mean [4] https://github.com/fivetran/benchmark
In A Bunch of Money on AWS and Some Benchmark Results
georgewfraser·Oct 10, 2017
It's not a limitation of the query tools. The problem is that users are running so many different queries against the same schema that you can't optimize for them all. Therefore, it's most important how a data warehouse performs when the optimization "tricks" don't work. Reasonable people can disagree on this last claim---we'll publish a v2 of this benchmark in a few months where we do more tuning https://github.com/fivetran/benchmark/issues/1
In Redshift vs. BigQuery vs. Snowflake benchmark

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.