HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
webrecorder
browsertrix-crawler
TypeScript2022–2023
webrecorder / browsertrix-crawler

browsertrix-crawler

Run a high-fidelity browser-based web archiving crawler in a single Docker container

Web DevelopmentSystems & InfrastructureDevOps & CloudAutomation & Workflows
Explore on GitHubRead Hacker News discussions
4Distinct people
5Mentions
0Explicit recommenders
1.2KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Run a high-fidelity browser-based web archiving crawler in a single Docker container

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 12:45:45 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month5 dated mentions

Observed mentions: 2022-04-11 23:00:06 UTC to 2023-08-01 19:12:07 UTC.

012022-04-01 00:00:00 UTC to 2022-05-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsApr 20222022-05-01 00:00:00 UTC to 2022-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-06-01 00:00:00 UTC to 2022-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-07-01 00:00:00 UTC to 2022-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-08-01 00:00:00 UTC to 2022-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-09-01 00:00:00 UTC to 2022-10-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-10-01 00:00:00 UTC to 2022-11-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2022-11-01 00:00:00 UTC to 2022-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-12-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsDec 20222023-01-01 00:00:00 UTC to 2023-02-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2023-02-01 00:00:00 UTC to 2023-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-03-01 00:00:00 UTC to 2023-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-04-01 00:00:00 UTC to 2023-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-05-01 00:00:00 UTC to 2023-06-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2023-06-01 00:00:00 UTC to 2023-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-07-01 00:00:00 UTC to 2023-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-08-01 00:00:00 UTC to 2023-09-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsAug 2023

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Apr 20222022-04-01 00:00:00 UTC2022-05-01 00:00:00 UTC101
May 20222022-05-01 00:00:00 UTC2022-06-01 00:00:00 UTC000
Jun 20222022-06-01 00:00:00 UTC2022-07-01 00:00:00 UTC000
Jul 20222022-07-01 00:00:00 UTC2022-08-01 00:00:00 UTC000
Aug 20222022-08-01 00:00:00 UTC2022-09-01 00:00:00 UTC000
Sep 20222022-09-01 00:00:00 UTC2022-10-01 00:00:00 UTC000
Oct 20222022-10-01 00:00:00 UTC2022-11-01 00:00:00 UTC101
Nov 20222022-11-01 00:00:00 UTC2022-12-01 00:00:00 UTC000
Dec 20222022-12-01 00:00:00 UTC2023-01-01 00:00:00 UTC000
Jan 20232023-01-01 00:00:00 UTC2023-02-01 00:00:00 UTC101
Feb 20232023-02-01 00:00:00 UTC2023-03-01 00:00:00 UTC000
Mar 20232023-03-01 00:00:00 UTC2023-04-01 00:00:00 UTC000
Apr 20232023-04-01 00:00:00 UTC2023-05-01 00:00:00 UTC000
May 20232023-05-01 00:00:00 UTC2023-06-01 00:00:00 UTC101
Jun 20232023-06-01 00:00:00 UTC2023-07-01 00:00:00 UTC000
Jul 20232023-07-01 00:00:00 UTC2023-08-01 00:00:00 UTC000
Aug 20232023-08-01 00:00:00 UTC2023-09-01 00:00:00 UTC101

What Hacker News says

1 sources
_a9·Oct 7, 2022
Not related to the OP topic or zim but I was looking into archiving my bookmarks and other content like documentation sites and wikis. I'll list some of the things I ended up using. ArchiveBox[1]: Pretty much a self-hosted wayback machine. It can save websites as plain html, screenshot, text, and some other formats. I have my bookmarks archived in it and have a bookmarklet to easily add new websites to it. If you use the docker-compose you can enable a full-text search backend for an easy search setup. WebRecorder[2]: A browser extension that creates WACZ archives directly in the browser capturing exactly what content you load. I use it on sites with annoying dynamic content that sites like wayback and ArchiveBox wouldn't be able to copy. ReplayWeb[3]: An interface to browse archive types like WARC, WACZ, and HAR. The interface is just li
Read the rest
ke browsing through your browser. It can be self-hosted as well for the full offline experience. browsertrix-crawler[4]: A CLI tool to scrape websites and output to WACZ. Its super easy to run with Docker and I use it to scrape entire blogs and docs for offline use. It uses Chrome to load webpages and has some extra features like custom browser profiles, interactive login, and autoscroll/autoplay. I use the `--generateWACZ` parameter so I can use ReplayWeb to easily browse through the final output. For bookmark and misc webpage archiving then ArchiveBox should be more than enough. Check out this repo for an amazing list of tools and resources https://github.com/iipc/awesome-web-archiving [1] https://github.com/ArchiveBox/ArchiveBox [2] https://webrecorder.net [3] https://replayweb.page [4] https://github.com/webrecorder/browsertrix-crawler
In How to download all of Wikipedia onto a USB flash drive

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.