HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
webrecorder
pywb
Python2016–2020
webrecorder / pywb

pywb

Core Python Web Archiving Toolkit for replay and recording of web archives

8Distinct people
10Mentions
0Explicit recommenders
1.7KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Core Python Web Archiving Toolkit for replay and recording of web archives

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 12:45:45 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year10 dated mentions

Observed mentions: 2016-07-09 17:12:35 UTC to 2020-07-31 05:08:17 UTC.

0123452016-01-01 00:00:00 UTC to 2017-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments20162017-01-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments20182019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 5 mentions; 0 submissions, 5 comments2020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2020

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20162016-01-01 00:00:00 UTC2017-01-01 00:00:00 UTC101
20172017-01-01 00:00:00 UTC2018-01-01 00:00:00 UTC101
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC202
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC505
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC101

What Hacker News says

5 sources
It depends how deep you want to go into the web archiving rabbit-hole, but using something like `archivenow`[0] would probably be the local optimum in terms of convenience, robustness, and permanence. It's "a tool to push web resources into web archives", which basically entails making requests of archiving services (like the Wayback machine, for example). Using `archivenow --all <url>`, you can back up <url> to a whole bunch of different services, and get links to snapshots of that page at the time of access. This is mostly sufficient, so long as at least one service is trustworthy and doesn't shut down unexpectedly. One issue that might come up is that the archives might miss content if the user has to interact with the page a bunch in order to reveal it, but that's unlikely to be an issue for the sorts of things an academic would be ar
Read the rest
chiving. As another poster mentioned, some bibliography managers have this capability baked in, or can acquire it with a plugin. However, if you want more control, you might consider using something like `pywb`[1] to make your own web archive, and then host it yourself (e.g., on S3 or something). For more robustness, you could also publish those WARC files as torrents, or on IPFS, or if you're really cool, as part of the transaction messages on some blockchain (provided that you can stomach the transaction fees and don't anticipate having a large audience in China). The above will probably suffice so long as the Internet continues to function as a going concern. To accommodate concerns about what happens in the case of total societal collapse, I've been backing up the papers that I cite by converting to binary[2] and preserving the result via aluminum punchcards, which I then mail to Sam Altman since he's got a way better game plan for the end times than I do[3]. I'm not sure if aluminum is optimal for this purpose though, both for material science reasons and because I've been informed that I am under investigation for my role in some sort of aluminum smuggling ring. If that fails, my last backup plan entails constructing an allegorical representation of the papers in question (which is surprisingly easy-- the Hero's Journey from Joseph Campbell[4]'s exegesis is basically about differential equations anyways), which I will then craft into a compelling mythology and teach to the local youths. Hopefully this will ensure that the citations are preserved via oral tradition even in the wake of cataclysmic change. Figuring out the BibTeX for this archival format will be pretty tricky, though. But yeah, unless you have some extra aluminum lying around, `archivenow` or whatever's built-in to your reference manager should do the trick. --- 0. github.com/oduwsdl/archivenow 1. github.com/webrecorder/pywb 2. After stripping most of the markup removing images using `pandoc`, as aluminum is expensive. 3. https://www.bloomberg.com/features/2018-rich-new-zealand-doo... 4. https://en.wikipedia.org/wiki/Joseph_Campbell
In Open Questions about Generative Adversarial Networks
I arrived a little late here, but for anyone just getting into web archiving, it's a fascinating space! We maintain a master community index with a list of all the major archiving organizations and open source tools here: https://github.com/pirate/ArchiveBox/wiki/Web-Archiving-Comm... Some of the cooler projects apart from ArchiveBox.io: - https://github.com/webrecorder/pywb / webrecorder.io - https://github.com/HelloZeroNet/ZeroNet - https://github.com/oduwsdl/ipwb - https://getpolarized.io - https://github.com/internetarchive/brozzler And many more blog posts, articles, projects, and organizations on the wiki!
In Why there’s so little left of the early internet

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.