HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
minimaxir
get-all-hacker-news-submissions-comments
Python2015–2018
minimaxir / get-all-hacker-news-submissions-comments

get-all-hacker-news-submissions-comments

Simple Python scripts to download all Hacker News submissions and comments and store them in a PostgreSQL database.

2Distinct people
13Mentions
0Explicit recommenders
127GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Simple Python scripts to download all Hacker News submissions and comments and store them in a PostgreSQL database.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 12:01:57 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year13 dated mentions

Observed mentions: 2015-05-11 14:59:02 UTC to 2018-02-19 05:26:05 UTC.

024682015-01-01 00:00:00 UTC to 2016-01-01 00:00:00 UTC (exclusive): 8 mentions; 2 submissions, 6 comments20152016-01-01 00:00:00 UTC to 2017-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments20162017-01-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2018

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20152015-01-01 00:00:00 UTC2016-01-01 00:00:00 UTC826
20162016-01-01 00:00:00 UTC2017-01-01 00:00:00 UTC202
20172017-01-01 00:00:00 UTC2018-01-01 00:00:00 UTC202
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC101

What Hacker News says

2 sources
Not quite what you asked: https://github.com/minimaxir/get-all-hacker-news-submissions... source: https://news.ycombinator.com/item?id=14337275 search related: https://hn.algolia.com/?query=bigquery-public-data:hacker_ne... found: https://news.ycombinator.com/item?id=14580698 (New HN data dump available with over 14.5m entries) 7.3GB through 2017Jun) https://news.ycombinator.com/item?id=10440502 (Tell HN: Full Hacker News dataset now available on BigQuery)
In Ask HN: Is there a script to download past 3 months of HN or top n posts?
Direct URL to project details: https://devpost.com/software/tagger-news A few comments: 1. To other commenters, as with the HN Vue demo a week ago (https://news.ycombinator.com/item?id=14284877), the project is a technical proof-of-concept; the aesthetics aren't the primary focus. 2. The Algolia API is better for scraping because it allows for bulk requests, unlike the official API (my old 2014 script still works I think: https://github.com/minimaxir/get-all-hacker-news-submissions...) 3) How much time did it take to manually label the training/test set before training the RF classifier? Even with topic modeling for extrapolating tags, accurate labeling for 20,000 submissions is a task.
In Tagger News takes a subset of HN articles, analyzes using ML, and applies tags

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.