internetarchive

Pick the items you want to mirror and seed them via their torrent file. https://help.archive.org/help/archive-bittorrents/ https://github.com/jjjake/internetarchive https://archive.org/services/docs/api/internetarchive/cli.ht... u/stavros wrote a design doc for a system (codename "Elephant") that would scale this up: https://news.ycombinator.com/item?id=45559219 (no affiliation, I am just a rando; if you are a library, museum, or similar institution, ask IA to drop some racks at your colo for replication, and as always, don't forget to donate to IA when able to and be kind to their infrastructure)

PSA: if you have a collection or other artifacts for ingest by IA, I’ll cover reasonable shipping costs to get them there. Above a certain size, they’ll handle logistics of packing and shipping for ingest. https://help.archive.org/help/how-do-i-make-a-physical-donat... Tools to make this easy exist if you already have digital versions. https://github.com/jjjake/internetarchive And don’t forget to send a few dollars if and when you can. https://archive.org/donate (no affiliation, I just like the public good)

https://help.archive.org/help/collections-a-basic-guide/ https://help.archive.org/help/how-to-upload-files-to-create-... https://help.archive.org/help/category/archive-org/uploading... https://help.archive.org/help/uploading-tips/ https://github.com/jjjake/internetarchive (for bulk uploading of items)

https://github.com/palewire/savepagenow https://github.com/jjjake/internetarchive The Internet Archive cannot trust arbitrary content previously archived, so it is more optimal to have whatever archival tools or operations you’re performing to make a request to Wayback to take a snapshot at the same time. If you’re bookmarking something, archive it too!

https://help.archive.org/help/how-to-upload-files-to-create-... https://github.com/jjjake/internetarchive might be helpful If you have an artifact in digital format already, you can self serve and don't need my help. Patron services at the org can provide guidance if you are unsure of item value, upload mechanics, collections, metadata, etc.

https://github.com/jjjake/internetarchive/commit/952ace47e0e... Me too, first commit was a bit more than 11 years ago.

Github: https://github.com/jjjake/internetarchive

> Can we talk about how cool the Wayback Machine Compare feature is? I consider myself an Internet Archive power-user; I spend hours playing with CDX queries, t̶r̶o̶l̶l̶i̶n̶g̶ trawling the archive for interesting tidbits that have been lost to time. I have spent countless evenings building scripts based on the internetarchive (https://github.com/jjjake/internetarchive) and iamine (https://github.com/jjjake/iamine) tools (along with a host of others). I am utterly ashamed I had never heard of the /diff/ feature between pages until I read your article. Thank you for bringing this to my attention! I am continuously impressed by the work they do at IA.

https://github.com/jjjake/internetarchive https://archive.org/services/docs/api/internetarchive/

I think this should fix that: https://github.com/jjjake/internetarchive/commit/1ac200cbbbe... This change will be in the next release, v1.8.5.

Sure, you can use `git bundle` to export your repository to a file, then upload it to the archive. There's a nice command-line tool for doing the upload (https://github.com/jjjake/internetarchive), so you can easily script this so that it's automatic.

I can't speak formally for the Internet Archive, but the existing content and services are not going to disappear overnight: funding comes from several sources, thought has been put in to organizational structure, and things have been designed to keep core access and preservation infrastructure running with minimal cost and effort (eg, if the economy tanks). Getting the content coverage people sometimes assume we already have is another matter. Additional funding (thanks for you donation!) go towards additional crawling and keeping up with the endless treadmill of media types and protocols. Eg, headless browser crawling development and deployment to capture javascript-heavy sites (https://github.com/internetarchive/brozzler); this is much more expensive than "classic" crawling. For more on increasing storage costs and the under-funded sta

I do wish somebody would crank out an easy to use desktop GUI tool for this. edit this will make it better https://gareth.halfacree.co.uk/2013/04/bulk-downloading-coll... and https://github.com/jjjake/internetarchive

Yes. This isn't the first time I've tried to contact toomuchtodo. Anywho, the amount of data is actually ~19GB which is well within what I can upload with my home connection. Unfortunately the ia tool is failing for me: https://github.com/jjjake/internetarchive/issues/176 Also, it's not really about the $3, more that a tonne of "$3" projects really add up over a year or so.