git-filter-repo

78/reduce-git-repos... https://stackoverflow.com/questions/3119850/is-there-a-way-t... https://stackoverflow.com/questions/38789265/git-delete-some... https://stackoverflow.com/questions/16057391/git-free-disk-s... https://stackoverflow.com/questions/31423525/how-to-reduce-d... https://stackoverflow.com/questions/16854425/compact-reposit... https://stackoverflow.com/questions/13999191/trimming-huge-g... https://stackoverflow.com/questions/4515580/how-do-i-remove-... These methods are all uniquely terrible in various ways. Most likely user error on my part. I need this technique: 1. Choose a range of commit hashes (or hashes before a commit) and remove them. This can be useful when splitting repos, for example on projects that started as backend+frontend where the frontend is being forked off in a new repo and the older backend portion needs to be removed from it for security/privacy. 2. Rebase all branches (including those that crossed the deleted portion) to preserve their structure but start/end as recently as possible. Optionally discard branches that were created and merged entirely within the deleted ported, unless they're the trunk of other branches that merge after the deleted portion. 3. Search for old commit hashes in commit messages and update them to the new hashes while rebasing. 4. Bonus points for updating stashes (or other git features) having any commit hashes in their names. Also for importing/exporting a list of important commit hashes for use in project management, such as updating hashes in comments on kanban boards like Jira. 5. More bonus points for searching for large files (such as app.js or other build artifacts) so that they can be stripped from commits in branches, preferably not on a main trunk like master. If you followed this far, I could also use a technique that rebases merged branches so that they form a series of D shapes instead of overlapping B shapes (this is useful during git bisect). Ideally this would happen automatically or be enforced via rules on sites like GitHub and GitLab. I always rebase my branches before merging, but others can't be bothered. https://www.atlassian.com/git/tutorials/merging-vs-rebasing Where I'm going with this: I git reset and git cherry-pick constantly in my own branches before merging, so that each branch has a clean work history like the trunk. I think of this as quantum committing, because I keep exploring the problem space until I find a solution that collapses (merges) into the history. The problem is that git GUIs are inadequate for this work. I need to be able to cut/copy/paste commits, drag and drop them for reordering, etc. It should also derive the commit diff needed to make a commit match a branch (or folder) rather than throwing a conflict in my face, so that it operates more like Apple's Time Machine. If I had this app, I could simply select all commits that I wanted to delete, it would ask me "this rewrites history, are you sure?", and then delete them and do the right thing for affected branches. It would also have infinite undo powered by git reflog. The idea being that commit hashes should not take priority - it's all about the information. We should never be trapped by the state of the repo, because that creates anxiety. So we're missing a tool to orchestrate git the way that Kubernetes orchestrates Docker.

vercoder9, but is slightly more sophisticated.) Note that this function will be very slow on large repositories. I can't think of a way to make it faster without writing actual code. (My only tip is that you can search for multiple targets at once by changing the grep command to `grep -E -q "$target"` and then using `git_find_references "HASH1|HASH2|HASH3"`.) Once you find the referring objects, you can use `git cat-file -p <hash>` on them to see their content, and you can repeatedly invoke this function to walk up a directory tree until you get to the enclosing commit. --- To actually fix the corruption, you have a couple of options: - If the corruption is in a branch that you don't care about, just delete the branch. - Find a good copy of the object, e.g. from another repository (you said the object did not appear, but I just want to point out that you should check using `git cat-file -p <hash>` rather than by looking in .git/objects/, because objects can also be stored in packs). Or you might be able to work out what the content was (if the corrupt object is a blob, aka file, you might be able to work out the content based on the previous and next versions of that file; `git hash-object <file>` will tell you if it's exactly right, but won't give you any hints if it's slightly wrong). - Rewrite the history to remove/replace the corrupt object. You can use a tool such as `git-filter-repo` (docs: https://htmlpreview.github.io/?https://github.com/newren/git..., in particular see --strip-blobs-with-ids). You can also use `git replace` to temporarily replace the corrupt object with a placeholder; it won't fix the commit history, but it might be useful if the corruption causes commands to fail.

s on Git often. Once you mention the "staging area" and "index", you'll usually lose them. I can't blame them...they just want to track their work!) You then try to run a `git filter-branch` command with the `--index-filter` since this can work against individual SHAs. Immediately, you get this: $: git filter-branch --index-filter '' WARNING: git-filter-branch has a glut of gotchas generating mangled history rewrites. Hit Ctrl-C before proceeding to abort, then use an alternative filtering tool such as 'git filter-repo' (https://github.com/newren/git-filter-repo/) instead. See the filter-branch manual page for more details; to squelch this warning, set FILTER_BRANCH_SQUELCH_WARNING=1. And then it just _hangs._ Since you know that going through six months of history can take a while, you think that it's working in the background. Nope, it just hangs there, waiting for you to CTRL-C instead of exiting with a non-zero exit code like EVERY OTHER CLI IN EXISTENCE. You go back into the `man` page to better understand this warning. Surprise surprise, `FILTER_BRANCH_SQUELCH_WARNING` isn't documented! So you provide the environment variable (if you know how, which isn't a given for junior engineers) and it finally runs...except you have no idea whether commits were rewritten or not (it does tell you whether the commits were rewritten, but you kind-of have to understand Git to understand the message). Now let's say you actually powered through and got it to run...but it caused you to lose work! (Remember, you have to `git push --force` or `git push --force-with-lease` to apply your changes, so this is stupid dangerous already, but it must be done.) To get it back, you then need to learn about `git reflog` and `git reset --hard`, but since `git filter-branch` runs against every commit that matched your pattern, you have to spend time scrolling back at the reflog to find the one you want to reset to. `bfg` exists to make this "simpler," but (a) it's a Java application, and you need to install the JRE to make it work, and (b) IT DOESN'T COME WITH GIT. Again, since this is a really common use-case with Git, it would be really nice if `git` had something more user-friendly, like: `git remove-references-to --regexp "PASSWORD" SHA_PATTERN` and a convenient message after the deed is done, like: 85 commits rewritten; run "git reset --hard [SHA_BEFORE_THE_CHANGES]" to undo. (Note: You must run `git push --force-with-lease to sync changes with your HTTPS-backed remote, "NAME_OF_REMOTE") However, since Git is maintained primarily for Linux development (afaict), this would most likely have to come from a third-party Git CLI. (I can already feel the rage from Linus for a proposal like this.) Funny enough, making things easier against the desires of protocol/system maintainers is how decentralized systems eventually centralize. I can see a world where everyone uses `gh` instead of `git` or has `git` aliased to `gh`, which means that a single company has de-facto control over the future of Git...