ReadableWebProxy

Obligatory bump for my project ReadableWebProxy (https://github.com/fake-name/ReadableWebProxy) that was originally intended to do this. At this point, it does a GIANT pile of additional things most of which are specific to my interests, but I think it might be at least marginally interesting to others. It does both full autonomous web-spidering of sites you specify, as well as synchronous rendering (You can browse other sites through it, with it rewriting all links to be internal links, and content for unknown sites fetched on-the-fly). I solve the javascript problem largely by removing all of it from the content I forward to the viewing client, though I do support remote sites that load their content through JS via headless chromium (I wrote a library for managing chrome that exposes the entire debugging protocol here: https://github.co

Most of that seems oriented at full, manual archival rather than automated (lightweight-ish) text caching for search operations. ArchiveBox (https://github.com/pirate/ArchiveBox) looks promising, although it isn't capturing the "live" browser feed. Readable-Web Proxy (https://github.com/fake-name/ReadableWebProxy) also looks interesting, but seems to be external as well. Maybe I just need to suck it up and roll my own.

Back in the mists of time, I used to use wwwoffle proxy. It was great for low-latency links, but also had the benefit of keeping an offline archive of whatever you'd browsed. Project's still there, although not sure how well it does with the modern web. http://www.gedanken.org.uk/software/wwwoffle/ There are a bunch of more modern variations too: https://archivebox.io/ - "Your own personal internet archive" https://getpolarized.io/ (as seen on HN previously) https://github.com/kanishka-linux/reminiscence https://github.com/fake-name/ReadableWebProxy

I read almost exclusively on a x86 device: A Windows Tablet. This is almost entirely because Windows 8 has the best touch-screen browser available on any platform: Internet Explorer Metro Edition (from a UX perspective, the actual browser internals are horrible). I've also written all my own server-side componentry[1]. 1: https://github.com/fake-name/ReadableWebProxy