ipwb

For my 80k+ bookmarks I use buku. Everything goes in there. It`s just a sqlite database (and buku is also a library for python). Good resources are saved in archivebox.io and are searchable via `rga`. In order to access my bookmarks i either need a local copy or have access to where my stuff is stored. To open any bookmark i search with `fzf` outside the browser. so i can work browser independently. (Can be integrated in rofi or dmenu.) And in the near future I`ll upload resources in a webarchive format to ipfs node to preserve some of the current internet (and to not get involved with rate limiting when I update my buku metadata. Sorry HN, I'm not spamming, just updating meta data for my bookmark archive.) https://github.com/jarun/Buku https://archivebox.io/ https://github.com/oduwsdl/ipwb [edited1 for formatting] [edited2 forgot to rela

How is this hosted on IPFS? Is it via ipwb? (https://github.com/oduwsdl/ipwb)

The largest active project doing this (to my knowledge) is the Inter-Planetary Wayback Machine: https://github.com/oduwsdl/ipwb There have been many other attempts though, including internetarchive.bak on IPFS, which ended up failing because it was too much data. http://archiveteam.org/index.php?title=INTERNETARCHIVE.BAK/i... http://brewster.kahle.org/2015/08/11/locking-the-web-open-a-...

I arrived a little late here, but for anyone just getting into web archiving, it's a fascinating space! We maintain a master community index with a list of all the major archiving organizations and open source tools here: https://github.com/pirate/ArchiveBox/wiki/Web-Archiving-Comm... Some of the cooler projects apart from ArchiveBox.io: - https://github.com/webrecorder/pywb / webrecorder.io - https://github.com/HelloZeroNet/ZeroNet - https://github.com/oduwsdl/ipwb - https://getpolarized.io - https://github.com/internetarchive/brozzler And many more blog posts, articles, projects, and organizations on the wiki!

Yup, that's the idea behind projects like https://github.com/HelloZeroNet/ZeroNet and https://github.com/oduwsdl/ipwb.

https://www.archiveteam.org/index.php?title=Wget_with_WARC_o... https://github.com/oduwsdl/ipwb #!/bin/sh cd ${HOME}/data/archive user_agent="Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27" domain="${1}" wget \ --mirror \ --page-requisites \ --restrict-file-names=nocontrol \ --timeout 60 \ --tries 5 \ --wait 1 \ --waitretry 5 \ --warc-cdx \ --warc-file="${domain}" \ --warc-header "operator: Wayback Mesh" \ -U "${user_agent}" \ -e robots=off \ "http://${domain}" ipwb index "${domain}.warc" > "${domain}.ipfs.cdxj"

You might also be interested in https://github.com/oduwsdl/ipwb, the author of webrecorder is a major contributor to this.

Hi jpalomaki, Have you heard of InterPlanetary Wayback (ipwb)? https://github.com/oduwsdl/ipwb InterPlanetary Wayback (ipwb) facilitates permanence and collaboration in web archives by disseminating the contents of WARC files into the IPFS network.