got

I've heard that jj has support for non-git backends? Can anyone comment on how difficult it would be to add support for another backend, any docs or examples? I have a project[0] that does the large file thing well, but is missing most of the version control porcelain. I've been looking for the path of least resistance to integrate it into something with a larger user base. [0] https://github.com/gotvc/got

I'm working on Got, version control like Git, but for files and directories of any size and E2E encrypted. https://github.com/gotvc/got Got is built on Blobcache, which is a general-purpose transactional storage layer and E2E encrypted backend. https://blobcache.io https://github.com/blobcache/blobcache

Yes the author reached out. There has not yet been a confusion among real users that I am aware of. https://github.com/gotvc/got/issues/20

> The problem is when you move beyond text files it gets hard to tell what changes between two versions without opening both versions in whatever program they come from and comparing. Yeah, totally agree. Got has not solved conflict resolution for arbitrary files. However, we can tell the user where the files differ, and that the file has changed. There is still value in being able to import files and directories of arbitrary sizes, and having the data encrypted. This is the necessary infrastructure to be able to do distributed version control on large amounts of private data. You can't do that easily with Git. It's very clunky even with remote helpers and LFS. I talk about that in the Why Got? section of the docs. https://github.com/gotvc/got/blob/master/doc/1.1_Why_Got.md

Me too. Version control is great, it should get more use outside of software. https://github.com/gotvc/got Notable differences: E2E encryption, parallel imports (Got will light up all your cores), and a data structure that supports large files and directories.

I work on a FOSS project in this space, Blobcache. https://github.com/blobcache/blobcache Trusting a server to store an application's state is a different thing from trusting it to author changes or to read the data. Servers should become dumber, and clients should become smarter. When I use an app, I want the app to load E2E encrypted state from storage (possibly on another machine, possibly not owned by me) make whatever changes and produce new encrypted data to send back to the server. The server should just be trusted for durability, and to prevent unauthorized access, but not to tell the truth about doing either of those things. Blobcache provides an API to facilitate transactions on E2EE state between a dumb storage server and any smart client. Blobcache can be installed on old hardware along with a VPN like Tailscale and then loade

I'm working on Blobcache. https://github.com/blobcache/blobcache Blobcache is content addressed storage, available over the network. Blobcache allow nodes to securely produce and consume storage. Configuration in similar to SSH, drop a public key in the configuration, and you're done. Blobcache is a universal backend for E2E encrypted applications. Docs - https://github.com/blobcache/blobcache/blob/master/doc/0.0_B... I'm also working on Got Version Control https://github.com/gotvc/got Got uses Blobcache for storing file data. Got is like Git, if you fixed all the problems with storing large files and directories in Git. There's no "large files provider" to configure separately. All the data for a commit goes to the same place. Got also encrypts all the data you put in it, E2E. If you've run into problems putting your whole home director

I've not heard the term "probabilistic tree" and I've having difficulty pulling up references. I suspect it's implemented by subpackage ptree[0]. Could you explain what makes probabilistic trees different from hash tables or other similar data structures? [0] https://github.com/gotvc/got/tree/master/pkg/gotkv/ptree

Sure, Git stores data in a trie. Each file is one blob identified by hash, and directories (called trees in Git) are blobs where each line is a directory entry with a name and the hash of a file or another tree. This means that modifying an object /down/a/long/path/like/this.txt has to create copies of all the trees on the way up. The technical term for this is "write amplification", and in Git it is affected by path length among other things. Got stores data in a probabilistic tree (GotKV[0]). The number of nodes before you get to data will scale logarithmically with the size of the entire filesystem, not the depth of a specific object. Then there is the issue of large files. A file in Git is always 1 blob. Syncing a large blob is not easy because if you are interrupted and have to restart, you have lost all your progress. You can'

This is a very similar to one of my projects "Got". https://github.com/gotvc/got The algorithms it uses are superior to rsync and git in a few ways. It comes short on features, especially for software development compared to Git. The motivation is more for personal file storage. I notice you're using Go and AGPL licensed, so you could borrow any of Got's libraries without issue. (Got is GPL licensed.) Definitely reach out in a GitHub issue.

FastCDC is the same chunking algorithm used in Got. https://github.com/gotvc/got

Thanks for sharing RocketGit. This is the first time I've heard of it, and yes, it does look like a cool copyleft solution to self-hosted Git. Another interesting option is Brendan Caroll's got[0], which allows sharing of repositories over INET256[1]. I'm sure there are other P2P approaches to Git, but this one recently piqued my interest. Unfortunately, it has a confusing and unresolved naming[2] conflict with OpenBSD's got (Game of Trees)[3]. [0] https://github.com/gotvc/got [1] https://github.com/inet256/inet256 [2] https://github.com/gotvc/got/issues/20 [3] https://gameoftrees.org/

I work on a project which solves a similar use case. https://github.com/gotvc/got Got also does E2E encryption, but it can additionally encrypt branch names from remote servers. It also handles syncing large files and directories better thanks to an improved data structure.

I've been working on a project "Got". Which deals with the LFS problem, mentioned in the post. https://github.com/gotvc/got Got isn't really trying to do software version control better than Git. It's trying to make general purpose file versioning practical, with a workflow similar to Git's.

In the same space is the key-value store underlying Got: GotKV. https://github.com/gotvc/got/tree/master/pkg/gotkv It stores encrypted blobs in any content-addressed store, and provides a copy-on-write key-value store API.