bup

bup: It backs things up bup is a program that backs things up. It's short for "backup." Can you believe that nobody else has named an open source program "bup" after all this time? Me neither. Despite its unassuming name, bup is pretty cool. To give you an idea of just how cool it is, I wrote you this poem: Bup is teh awesome What rhymes with awesome? I guess maybe possum But that's irrelevant. Hmm. Did that help? Maybe prose is more useful after all. Reasons bup is awesome bup has a few advantages over other backup software: It uses a rolling checksum algorithm (similar to rsync) to split large files into chunks. The most useful result of this is you can backup huge virtual machine (VM) disk images, databases, and XML files incrementally, even though they're typically all in one huge file, and not use tons of disk space for multiple versions. It uses the packfile format from git (the open source version control system), so you can access the stored data even if you don't like bup's user interface. Unlike git, it writes packfiles directly (instead of having a separate garbage collection / repacking stage) so it's fast even with gratuitously huge amounts of data. bup's improved index formats also allow you to track far more filenames than git (millions) and keep track of far more objects (hundreds or thousands of gigabytes). Data is "automagically" shared between incremental backups without having to know which backup is based on which other one - even if the backups are made from two different computers that don't even know about each other. You just tell bup to back stuff up, and it saves only the minimum amount of data needed. You can back up directly to a remote bup server, without needing tons of temporary disk space on the computer being backed up. And if your backup is interrupted halfway through, the next run will pick up where you left off. And it's easy to set up a bup server: just install bup on any machine where you have ssh access. Bup can use "par2" redundancy to recover corrupted backups even if your disk has undetected bad sectors. Even when a backup is incremental, you don't have to worry about restoring the full backup, then each of the incrementals in turn; an incremental backup acts as if it's a full backup, it just takes less disk space. You can mount your bup repository as a FUSE filesystem and access the content that way, and even export it over Samba. It's written in python (with some C parts to make it faster) so it's easy for you to extend and maintain. Reasons you might want to avoid bup This is a very early version. Therefore it will most probably not work for you, but we don't know why. It is also missing some probably-critical features. It requires python >= 2.4, a C compiler, and an installed git version >= 1.5.3.1. It currently only works on Linux, MacOS X >= 10.4, NetBSD, Solaris, or Windows (with Cygwin). Patches to support other platforms are welcome. Getting started From source Check out the bup source code using git: git clone git://github.com/bup/bup Install the needed python libraries (including the development libraries). On Debian/Ubuntu this is usually sufficient (run as root): apt-get install python2.6-dev python-fuse apt-get install python-pyxattr python-pylibacl apt-get install linux-libc-dev Substitute python2.5-dev or python2.4-dev if you have an older system. Alternately, on newer Debian/Ubuntu versions, you can try this: apt-get build-dep bup On CentOS (for CentOS 6, at least), this should be sufficient (run as root): yum groupinstall "Development Tools" yum install python python-dev yum install fuse-python pyxattr pylibacl yum install perl-Time-HiRes In addition to the default CentOS repositories, you may need to add RPMForge (for fuse-python) and EPEL (for pyxattr and pylibacl). Build the python module and symlinks: make Run the tests: make test (The tests should pass. If they don't pass for you, stop here and send me an email.) You can install bup via "make install", and override the default destination with DESTDIR and PREFIX. Files are normally installed to "$DESTDIR/$PREFIX" where DESTDIR is empty by default, and PREFIX is set to /usr. So if you wanted to install bup to /opt/bup, you might do something like this: make install DESTDIR=/opt/bup PREFIX='' From binary packages Binary packages of bup are known to be built for the following OSes: Debian: http://packages.debian.org/search?searchon=names&keywords=bup Ubuntu: http://packages.ubuntu.com/search?searchon=names&keywords=bup pkgsrc (NetBSD, Dragonfly, and others) http://pkgsrc.se/sysutils/bup http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/sysutils/bup/ Using bup Try making a local backup as a tar file: tar -cvf - /etc | bup split -n local-etc -vv Try restoring your backup tarball: bup join local-etc | tar -tf - Look at how much disk space your backup took: du -s ~/.bup Make another backup (which should be mostly identical to the last one; notice that you don't have to specify that this backup is incremental, it just saves space automatically): tar -cvf - /etc | bup split -n local-etc -vv Look how little extra space your second backup used on top of the first: du -s ~/.bup Restore your old backup again (the ~1 is git notation for "one older than the most recent"): bup join local-etc~1 | tar -tf - Get a list of your previous backups: GIT_DIR=~/.bup git log local-etc Make a backup on a remote server (which must already have the 'bup' command somewhere in the server's PATH (see /etc/profile, etc/environment, ~/.profile, or ~/.bashrc), and be accessible via ssh. Make sure to replace SERVERNAME with the actual hostname of your server): tar -cvf - /etc | bup split -r SERVERNAME: -n local-etc -vv Try restoring the remote backup tarball: bup join -r SERVERNAME: l