HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
minimaxir
facebook-page-post-scraper
Python2015–2017
minimaxir / facebook-page-post-scraper · Archived

facebook-page-post-scraper

Data scraper for Facebook Pages, and also code accompanying the blog post How to Scrape Data From Facebook Page Posts for Statistical Analysis

4Distinct people
10Mentions
1Explicit recommenders
2.1KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Data scraper for Facebook Pages, and also code accompanying the blog post How to Scrape Data From Facebook Page Posts for Statistical Analysis

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

I love the way you format your py code. clean and super easy to read. https://github.com/minimaxir/facebook-page-post-scraper/blob...

johnwheeler used a recommendation phrase. One indexed comment matches. This is a phrase match, not a verdict.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 12:45:45 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month10 dated mentions

Observed mentions: 2015-07-24 14:21:41 UTC to 2017-05-05 05:53:17 UTC.

01232015-07-01 00:00:00 UTC to 2015-08-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsJul 20152015-08-01 00:00:00 UTC to 2015-09-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2015-09-01 00:00:00 UTC to 2015-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2015-10-01 00:00:00 UTC to 2015-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2015-11-01 00:00:00 UTC to 2015-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2015-12-01 00:00:00 UTC to 2016-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-01-01 00:00:00 UTC to 2016-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-02-01 00:00:00 UTC to 2016-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-03-01 00:00:00 UTC to 2016-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-04-01 00:00:00 UTC to 2016-05-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2016-05-01 00:00:00 UTC to 2016-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-06-01 00:00:00 UTC to 2016-07-01 00:00:00 UTC (exclusive): 3 mentions; 1 submissions, 2 commentsJun 20162016-07-01 00:00:00 UTC to 2016-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-08-01 00:00:00 UTC to 2016-09-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2016-09-01 00:00:00 UTC to 2016-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-10-01 00:00:00 UTC to 2016-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-11-01 00:00:00 UTC to 2016-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-12-01 00:00:00 UTC to 2017-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2017-01-01 00:00:00 UTC to 2017-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2017-02-01 00:00:00 UTC to 2017-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2017-03-01 00:00:00 UTC to 2017-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2017-04-01 00:00:00 UTC to 2017-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2017-05-01 00:00:00 UTC to 2017-06-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsMay 2017

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Jul 20152015-07-01 00:00:00 UTC2015-08-01 00:00:00 UTC110
Aug 20152015-08-01 00:00:00 UTC2015-09-01 00:00:00 UTC101
Sep 20152015-09-01 00:00:00 UTC2015-10-01 00:00:00 UTC000
Oct 20152015-10-01 00:00:00 UTC2015-11-01 00:00:00 UTC000
Nov 20152015-11-01 00:00:00 UTC2015-12-01 00:00:00 UTC000
Dec 20152015-12-01 00:00:00 UTC2016-01-01 00:00:00 UTC000
Jan 20162016-01-01 00:00:00 UTC2016-02-01 00:00:00 UTC000
Feb 20162016-02-01 00:00:00 UTC2016-03-01 00:00:00 UTC000
Mar 20162016-03-01 00:00:00 UTC2016-04-01 00:00:00 UTC000
Apr 20162016-04-01 00:00:00 UTC2016-05-01 00:00:00 UTC101
May 20162016-05-01 00:00:00 UTC2016-06-01 00:00:00 UTC000
Jun 20162016-06-01 00:00:00 UTC2016-07-01 00:00:00 UTC312
Jul 20162016-07-01 00:00:00 UTC2016-08-01 00:00:00 UTC000
Aug 20162016-08-01 00:00:00 UTC2016-09-01 00:00:00 UTC202
Sep 20162016-09-01 00:00:00 UTC2016-10-01 00:00:00 UTC000
Oct 20162016-10-01 00:00:00 UTC2016-11-01 00:00:00 UTC000
Nov 20162016-11-01 00:00:00 UTC2016-12-01 00:00:00 UTC000
Dec 20162016-12-01 00:00:00 UTC2017-01-01 00:00:00 UTC101
Jan 20172017-01-01 00:00:00 UTC2017-02-01 00:00:00 UTC000
Feb 20172017-02-01 00:00:00 UTC2017-03-01 00:00:00 UTC000
Mar 20172017-03-01 00:00:00 UTC2017-04-01 00:00:00 UTC000
Apr 20172017-04-01 00:00:00 UTC2017-05-01 00:00:00 UTC000
May 20172017-05-01 00:00:00 UTC2017-06-01 00:00:00 UTC101

What Hacker News says

2 sources
> (Disclosure: I have developed a Facebook Page Post Scraper [https://github.com/minimaxir/facebook-page-post-scraper] which explicitly follows the permissions set by the Facebook API.) I played with the idea of creating some social aggregation type service with some friends (as a business). The more I read about FB's past behavior with regard to this, and how essential they are to any sort of service like, that, I canned the project. Regardless of what their TOS say, if you get on their radar and they send you a cease-and-desist, it's game over. Facebook is not in the business of subverting their revenue stream, so if you are making money off them and it's preventing them from capitalizing on their users, don't expect to last long if you exist by their grace. Really, there's an interesting space between so small nobody cares and large
Read the rest
enough that getting shut down is a real problem. A lot of projects start small and end up (relatively) large, but without a good way to pay for the service itself. While not every service needs to be a business and make money, once you reach the level where you risk either being shut out of your data source or you need to somehow work out an understanding with that source, how do you approach that when being able to pay is off the table? Not to mention the problem approaching before you have to and forcing the situation, or waiting too long and risking the wrath of the source because you've abused their service as long as you have. Has anyone else been in this situation and found an approach that works?
In Web Scraping in 2016
Keep in mind that companies have sued for scraping not through the API, for example LinkedIn, which explicitly prevents scraping via the ToS: http://www.informationweek.com/software/social/linkedin-sues... OKCupid did a DMCA takedown for researchers releasing scraped data: https://www.engadget.com/2016/05/17/publicly-released-okcupi... Since both of these incidents, I now only scrape if it's a) through the API following rate limits or b) if there is no API, and the data has the explicit purpose of being shared publically (e.g blogs), I follow robots.txt. Of course, most companies have a do-not-scrape clause in their ToS anyways, to my personal frustration. (Disclosure: I have developed a Facebook Page Post Scraper [https://github.com/minimaxir/facebook-page-post-scraper] which explicitly follows the permissions set by the Facebook API.)
In Web Scraping in 2016

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.