DAdarth_avocado·Feb 20, 2024
The setup instructions mention what you’re asking.
https://github.com/HTTPArchive/httparchive.org/blob/main/doc...
In Warning: $14k BigQuery charge in 2 hoursHTTPArchive
httparchive.org
JavaScript2024–2024
HTTPArchive / httparchive.org
httparchive.org
The HTTP Archive website
3Distinct people
4Mentions
0Explicit recommenders
363GitHub stars
What to know before exploring
What it is
The HTTP Archive website
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 12:54:37 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by hour4 dated mentions
Observed mentions: 2024-02-20 21:21:24 UTC to 2024-02-20 21:49:53 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC. One observed period does not establish a trend.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Feb 20, 21:00 | 2024-02-20 21:00:00 UTC | 2024-02-20 22:00:00 UTC | 4 | 0 | 4 |
What Hacker News says
4 sourcesI am sorry but this seems to be more of a “TLDR; didn’t read;” situation. The http archive clearly mentions that the data is available for offline processing or for querying online on BQ. And in the “Getting started” section of the instructions, it is mentioned multiple times on how BQ will charge you. And even if it wasn’t mentioned anywhere, it’s a little presumptuous to assume a tool for processing data will not charge you money for literally processing TBs of data again and again.
> Note: BigQuery has a free tier that you can use to get started without enabling billing. At the time of this writing, the free tier allows 10GB of storage and 1TB of data processing per month. Google also provides a $300 credit for new accounts.
> Note: The size of the tables you query are important because BigQuery is billed based on the number of process
Read the rest
ed data. There is 1TB of processed data included in the free tier, so running a full scan query on one of the larger tables can easily eat up your quota. This is where it becomes important to design queries that process only the data you wish to explore
> When we look at the results of this, you can see how much data was processed during this query. Writing efficient queries limits the number of bytes processed - which is helpful since that's how BigQuery is billed. Note: There is 1TB free per month
https://github.com/HTTPArchive/httparchive.org/blob/main/doc...
To be honest, even the official guide [1] for BG does not have any information about how to make some info about query cost, budget, and service limits mechanisms [2].
I think the HTTP Archive team could set something in that regard.
PS: When I was an instructor for some cloud training in AWS, the first 2 hours were only to set up billing and budgets to avoid any kind of situation like this. No one would start training without all those locks in place in the first place.
[1] - https://github.com/HTTPArchive/httparchive.org/blob/main/doc...
[2] - https://cloud.google.com/bigquery/docs/best-practices-costs
In Warning: $14k BigQuery charge in 2 hoursThat is pretty clearly documented in the setup instructions.
https://github.com/HTTPArchive/httparchive.org/blob/main/doc...
There are multiple notes about cost. In particular, this one stands out.
> Note: The size of the tables you query are important because BigQuery is billed based on the number of processed data. There is 1TB of processed data included in the free tier, so running a full scan query on one of the larger tables can easily eat up your quota. This is where it becomes important to design queries that process only the data you wish to explore
In Warning: $14k BigQuery charge in 2 hours