torrent-net

I honestly think that the idea of letting people search inside a torrent before fully downloading it has been overlooked and might be the next big step in P2P file-sharing: https://github.com/lmatteis/torrent-net The idea is that you can actually query stuff inside a torrent; like actually perform an SQL query and only the pieces relevant to the query will be downloaded. This sort of allows for distributed querying if you think about it, since your query could be satisfied by many different peers that are seeding/leeching. Imagine a Web built this way. Where sites are served by people that visit them and are not just static sites, but fully queryable as you'd expect in the normal HTTP web. I wrote more on this here: https://medium.com/@lmatteis/torrentnet-bd4f6dab15e4

This is awesome. If only we could share this huge dump of PDF files as a more structured format, perhaps using SQLite [1, 2], we could search through the torrents without having to wait for all of them to download beforehand. Although I guess the fact of having to "download them all beforehand" forces the data to be spread across various computers, hence increases availability of the data. One idea I had regarding this is perhaps structuring the contents of torrents as "append-only binary trees". So as new dumps are released every month, one can simply start downloading the torrent and has "search capabilities" for new data as well. 1. https://github.com/lmatteis/torrent-net 2. https://www.youtube.com/watch?v=EKttt8PYu5M&feature=youtu.be