HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
chiphuyen
lazynlp
Python2019–2019
chiphuyen / lazynlp

lazynlp

Library to scrape and clean web pages to create massive datasets.

Web DevelopmentScience & EngineeringAI & Machine LearningAutomation & WorkflowsLibraries & SDKs
Explore on GitHubRead Hacker News discussions
5Distinct people
5Mentions
0Explicit recommenders
2.3KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Library to scrape and clean web pages to create massive datasets.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by week5 dated mentions

Observed mentions: 2019-02-28 01:38:02 UTC to 2019-06-29 06:18:55 UTC. Weeks start Monday, UTC.

01232019-02-25 00:00:00 UTC to 2019-03-04 00:00:00 UTC (exclusive): 3 mentions; 3 submissions, 0 commentsFeb 252019-03-04 00:00:00 UTC to 2019-03-11 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-03-11 00:00:00 UTC to 2019-03-18 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-03-18 00:00:00 UTC to 2019-03-25 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-03-25 00:00:00 UTC to 2019-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-04-01 00:00:00 UTC to 2019-04-08 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-04-08 00:00:00 UTC to 2019-04-15 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-04-15 00:00:00 UTC to 2019-04-22 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-04-22 00:00:00 UTC to 2019-04-29 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsApr 222019-04-29 00:00:00 UTC to 2019-05-06 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-05-06 00:00:00 UTC to 2019-05-13 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-05-13 00:00:00 UTC to 2019-05-20 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-05-20 00:00:00 UTC to 2019-05-27 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-05-27 00:00:00 UTC to 2019-06-03 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-06-03 00:00:00 UTC to 2019-06-10 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-06-10 00:00:00 UTC to 2019-06-17 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-06-17 00:00:00 UTC to 2019-06-24 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2019-06-24 00:00:00 UTC to 2019-07-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsJun 24

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Feb 252019-02-25 00:00:00 UTC2019-03-04 00:00:00 UTC330
Mar 42019-03-04 00:00:00 UTC2019-03-11 00:00:00 UTC000
Mar 112019-03-11 00:00:00 UTC2019-03-18 00:00:00 UTC000
Mar 182019-03-18 00:00:00 UTC2019-03-25 00:00:00 UTC000
Mar 252019-03-25 00:00:00 UTC2019-04-01 00:00:00 UTC000
Apr 12019-04-01 00:00:00 UTC2019-04-08 00:00:00 UTC000
Apr 82019-04-08 00:00:00 UTC2019-04-15 00:00:00 UTC000
Apr 152019-04-15 00:00:00 UTC2019-04-22 00:00:00 UTC000
Apr 222019-04-22 00:00:00 UTC2019-04-29 00:00:00 UTC000
Apr 292019-04-29 00:00:00 UTC2019-05-06 00:00:00 UTC000
May 62019-05-06 00:00:00 UTC2019-05-13 00:00:00 UTC000
May 132019-05-13 00:00:00 UTC2019-05-20 00:00:00 UTC000
May 202019-05-20 00:00:00 UTC2019-05-27 00:00:00 UTC000
May 272019-05-27 00:00:00 UTC2019-06-03 00:00:00 UTC000
Jun 32019-06-03 00:00:00 UTC2019-06-10 00:00:00 UTC000
Jun 102019-06-10 00:00:00 UTC2019-06-17 00:00:00 UTC000
Jun 172019-06-17 00:00:00 UTC2019-06-24 00:00:00 UTC110
Jun 242019-06-24 00:00:00 UTC2019-07-01 00:00:00 UTC101

What Hacker News says

5 sources
westurner·Jun 29, 2019
> I imagine it starts with "links -dump", but then there's using the title as the filename, The title tag may exceed the filename length limit, be the same for nested pages, or contain newlines that must be escaped. These might be helpful for your use case: "Newspaper3k: Article scraping & curation" https://github.com/codelucas/newspaper lazyNLP "Library to scrape and clean web pages to create massive datasets" https://github.com/chiphuyen/lazynlp/blob/master/README.md#s... scrapinghub/extruct https://github.com/scrapinghub/extruct > extruct is a library for extracting embedded metadata from HTML markup. > It also has a built-in HTTP server to test its output as JSON. > Currently, extruct supports: > - W3C's HTML Microdata > - embedded JSON-LD > - Microformat via mf2py > - Facebook's Open Graph > - (experimental) RDFa via rdflib
In Ask HN: Scripts/commands for extracting URL article text? (links -dump but)

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.