HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
internetarchive
warcprox
Python2014–2026
internetarchive / warcprox

warcprox

WARC writing MITM HTTP/S proxy

10Distinct people
11Mentions
0Explicit recommenders
468GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

WARC writing MITM HTTP/S proxy

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 12:56:33 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year11 dated mentions

Observed mentions: 2014-10-29 21:06:42 UTC to 2026-05-09 07:58:11 UTC.

01232014-01-01 00:00:00 UTC to 2015-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments20142015-01-01 00:00:00 UTC to 2016-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-01-01 00:00:00 UTC to 2017-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2017-01-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments20202021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments2023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20142014-01-01 00:00:00 UTC2015-01-01 00:00:00 UTC101
20152015-01-01 00:00:00 UTC2016-01-01 00:00:00 UTC000
20162016-01-01 00:00:00 UTC2017-01-01 00:00:00 UTC101
20172017-01-01 00:00:00 UTC2018-01-01 00:00:00 UTC101
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC101
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC101
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC101
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC000
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC303
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC101
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC000
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC000
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC101

What Hacker News says

3 sources
Since this is pretty new, some background: WARC is a file format written and read by a rather small but specialized set of web crawler tools, most notably the internet archive's tooling. For example, its java-based crawler heretrix produces warc files. There are a couple of other very cool tools, such as warcprox, which can create web archives from web browser activity by acting as a proxy, pywb which play back the same to make archived versions browsable, and related libraries [1](shoutout to Noah Levitt, Ilya Kreymer and collaborators for building all this). The file format itself is an iso standard, and it's very simple: Capture all http headers sent and received, and all http bodies sent and received, and do simple de-duplication based on digests of the body. There is a companion format, CDX, which builds indexes from warc files (whic
Read the rest
h in turn are just concatenated records, so rather robust). Although all of this is great, I worry a bit about where we're heading with QUIC / udp-based protocols, websockets and other very involved protocols which ultimately make archival much harder. If there's anything you can do to help these (or other) tools to keep our web's archival records alive and flowing, please do so. [1] https://github.com/internetarchive/warcprox
In Show HN: WarcDB: Web crawl data as SQLite databases
Wow, it's really rare these days to see a tool that supports WARC. Despite being an ISO standard [1] and the default archive format of the internet archive, and despite a handfull of lovingly crafted tools (such as webrecorder [2], warcprox etc.), it never seems to have caught on in a broader context. Really a shame - I' deeply convinced that the ability to archive and replay requests is a technique for defending and strengthening user rights. Links: [1] https://www.iso.org/standard/44717.html [2] https://github.com/webrecorder/webrecorder-desktop [3] https://github.com/internetarchive/warcprox
In Tofuproxy – Web proxy, TLS terminator, X.509 TOFU manager, WARC/gemini browser

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.