HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
mozilla
gcp-ingestion
Java2020–2022
mozilla / gcp-ingestion

gcp-ingestion

Documentation and implementation of telemetry ingestion on Google Cloud Platform

3Distinct people
3Mentions
0Explicit recommenders
91GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Documentation and implementation of telemetry ingestion on Google Cloud Platform

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month3 dated mentions

Observed mentions: 2020-03-21 11:00:09 UTC to 2022-03-18 21:56:50 UTC.

012020-03-01 00:00:00 UTC to 2020-04-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsMar 20202020-04-01 00:00:00 UTC to 2020-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2020-05-01 00:00:00 UTC to 2020-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2020-06-01 00:00:00 UTC to 2020-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2020-07-01 00:00:00 UTC to 2020-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2020-08-01 00:00:00 UTC to 2020-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2020-09-01 00:00:00 UTC to 2020-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2020-10-01 00:00:00 UTC to 2020-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2020-11-01 00:00:00 UTC to 2020-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2020-12-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-01-01 00:00:00 UTC to 2021-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-02-01 00:00:00 UTC to 2021-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-03-01 00:00:00 UTC to 2021-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsMar 20212021-04-01 00:00:00 UTC to 2021-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-05-01 00:00:00 UTC to 2021-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-06-01 00:00:00 UTC to 2021-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-07-01 00:00:00 UTC to 2021-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-08-01 00:00:00 UTC to 2021-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-09-01 00:00:00 UTC to 2021-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-10-01 00:00:00 UTC to 2021-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-11-01 00:00:00 UTC to 2021-12-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2021-12-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-01-01 00:00:00 UTC to 2022-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-02-01 00:00:00 UTC to 2022-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-03-01 00:00:00 UTC to 2022-04-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsMar 2022

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Mar 20202020-03-01 00:00:00 UTC2020-04-01 00:00:00 UTC110
Apr 20202020-04-01 00:00:00 UTC2020-05-01 00:00:00 UTC000
May 20202020-05-01 00:00:00 UTC2020-06-01 00:00:00 UTC000
Jun 20202020-06-01 00:00:00 UTC2020-07-01 00:00:00 UTC000
Jul 20202020-07-01 00:00:00 UTC2020-08-01 00:00:00 UTC000
Aug 20202020-08-01 00:00:00 UTC2020-09-01 00:00:00 UTC000
Sep 20202020-09-01 00:00:00 UTC2020-10-01 00:00:00 UTC000
Oct 20202020-10-01 00:00:00 UTC2020-11-01 00:00:00 UTC000
Nov 20202020-11-01 00:00:00 UTC2020-12-01 00:00:00 UTC000
Dec 20202020-12-01 00:00:00 UTC2021-01-01 00:00:00 UTC000
Jan 20212021-01-01 00:00:00 UTC2021-02-01 00:00:00 UTC000
Feb 20212021-02-01 00:00:00 UTC2021-03-01 00:00:00 UTC000
Mar 20212021-03-01 00:00:00 UTC2021-04-01 00:00:00 UTC000
Apr 20212021-04-01 00:00:00 UTC2021-05-01 00:00:00 UTC000
May 20212021-05-01 00:00:00 UTC2021-06-01 00:00:00 UTC000
Jun 20212021-06-01 00:00:00 UTC2021-07-01 00:00:00 UTC000
Jul 20212021-07-01 00:00:00 UTC2021-08-01 00:00:00 UTC000
Aug 20212021-08-01 00:00:00 UTC2021-09-01 00:00:00 UTC000
Sep 20212021-09-01 00:00:00 UTC2021-10-01 00:00:00 UTC000
Oct 20212021-10-01 00:00:00 UTC2021-11-01 00:00:00 UTC000
Nov 20212021-11-01 00:00:00 UTC2021-12-01 00:00:00 UTC101
Dec 20212021-12-01 00:00:00 UTC2022-01-01 00:00:00 UTC000
Jan 20222022-01-01 00:00:00 UTC2022-02-01 00:00:00 UTC000
Feb 20222022-02-01 00:00:00 UTC2022-03-01 00:00:00 UTC000
Mar 20222022-03-01 00:00:00 UTC2022-04-01 00:00:00 UTC101

What Hacker News says

3 sources
I don't know how many folks will see this, and of those that do I don't expect many will necessarily be moved by what I say here. I'm going to say it anyways, however, and then I may never look at this thread again. I'm the person who designed the download token scheme that is discussed in this article, and, while I understand all of the concerns and suspicions, I believe that the way we designed this and the way we handle our telemetry data means that this is not the privacy violation some of you are claiming it is. Also, to be clear, I am speaking for myself here, these are my own thoughts and opinions, and I am not representing Mozilla in any official capacity. So, a download token is a UUID associated with a unique download event. It gets generated when you click the 'download' link, added to the installer, and then passed through to
Read the rest
the installed browser. It is returned to us in the telemetry pings that the browser sends back to our telemetry ingestion endpoints. When the download happens, on the server side we capture the download token and the GA session ID and store those in a table. There is nothing else stored in this table. Having access to this table means that you can correlate the user's activity on the Mozilla website that GA provides with the telemetry data that Firefox sends us. The website activity contains URLs that the user visited, so we consider this "category 3" data (see https://wiki.mozilla.org/Data_Collection#Data_Collection_Cat...), quite sensitive. For that reason this table has highly restricted access, only a small number of individuals are able to get to it. Access restrictions offer no protection against subpoenas, of course. But I believe you can safely maintain your anonymity by opting out of our telemetry gathering, because when you opt out of telemetry we delete all of the historical telemetry data we have collected for your Firefox profile. Everything, including all of the records that contain the download token. If this happens, all we are left with is that original record with the download token and a GA session. The download token can no longer be correlated with your telemetry data, and we have no way of associating your Firefox installation with your GA session, not even under subpoena. And this is all assuming that you haven't blocked GA, or that you haven't specified 'Do Not Track' before visiting our website. If you've done either of those things, we won't have a GA session ID for you to begin with. Oh, incidentally, we never store any IP addresses or other PII in our telemetry data. That all gets scrubbed during ingestion. Again, I don't expect this to have much impact, but I'm sharing what I know to counter some of the more extreme claims that this removes the ability for Firefox users to remain anonymous. Finally, we have the obvious question: Why we would even do this? Believe it or not, understanding your user base does actually have some value in serving that user base. For most of Firefox's existence, there has been no trustable feedback loop. Sure, folks out there in the world have opinions, and share them, but opinions differ, and anecdotes are not data. If one person thinks most users will like a particular change, and someone else thinks they won't, nobody can prove their point in any meaningful way. The folks making decisions about Firefox have been flying blind. And, as many of you in this thread have pointed out, it hasn't necessarily been going that well. In Firefox's early years, there was lots of low hanging fruit, and the competition was a poorly maintained Internet Explorer, so it was easy to win a bunch of market share. Then Chrome came on the scene with their effectively limitless budget and famously data driven product process. We'll never match their budget, but we can try to make choices based on data instead of just letting whoever has the most organizational power decide. My team has spent the last few years building out a data infrastructure that we hope will support better decision making going forward while still trying to honor user privacy and choice. This is a tough balance to strike, and we're far from perfect, but we do our best. You can learn about or data collection infrastructure and policies in great detail on our docs site (https://docs.telemetry.mozilla.org/index.html), and you can see nearly all of the code that handles our data ingestion and processing in our public repositories (https://github.com/mozilla/gcp-ingestion and https://github.com/mozilla/bigquery-etl).
In Each Firefox download has a unique identifier
I used to work on Mozilla's data platform. That stuff is all open source. See e.g. https://github.com/mozilla/gcp-ingestion/ for the ingestion pipeline, https://github.com/mozilla/bigquery-etl for queries/ETL, and https://github.com/mozilla/looker-spoke-default/ for looker model definitions for that data. Also go read the docs at https://docs.telemetry.mozilla.org/. Those will give you insights into every way they use data. I've never seen a company that's more open about their data usage.
In Firefox is the alternative to a Chrome hegemony

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.