notebooks

Some resources to get you started: - How to use the Hacker News dataset https://medium.com/google-cloud/big-data-stories-in-seconds-... - Discussion of the HN dataset announcement here https://news.ycombinator.com/item?id=10440502 - An iPython notebook: https://github.com/fhoffa/notebooks/blob/master/analyzing%20... - More: http://debarghyadas.com/writes/looking-back-at-9-years-of-ha... Disclaimer: I'm Felipe Hoffa, and I work at Google. (https://twitter.com/felipehoffa)

I charted the peak times here: https://github.com/fhoffa/notebooks/blob/master/analyzing%20... (Python notebook - renders well on desktop, but GitHub might not show a nice rendering if you try it on mobile)

Hi! I'm the one that loaded this dataset into BigQuery. Feel free to ask any questions :). The notebook with sample queries and visualizations: https://github.com/fhoffa/notebooks/blob/master/analyzing%20...

Clickables: https://bigquery.cloud.google.com/table/fh-bigquery:hackerne... https://github.com/fhoffa/notebooks/blob/master/analyzing%20... https://github.com/minimaxir/get-all-hacker-news-submissions... http://minimaxir.com/2015/10/reddit-bigquery/

https://bigquery.cloud.google.com/table/fh-bigquery:hackernews.comments The dataset is up-to-date for October 2015 and uses the official HN API as a data source. (Unfortuntely, this includes the HTML formatting in the comments) Dataset is about 4GB total; since BigQuery allows for 1000GB processing for free each month, it is effectively no cost to analyze. Felipe Hoffa (who uploaded the dataset) has links to the dataset along with some sample Python code for analysis: https://github.com/fhoffa/notebooks/blob/master/analyzing%20hacker%20news.ipynb I have a few scripts for downloading all the data manually (https://github.com/minimaxir/get-all-hacker-news-submissions-comments ), but the BigQuery table may be more pragmatic. I can do some more sample queries if requested. (See my BigQuery tutorial for Reddit data: http://minimaxir.com/2015/1