Backup strategy for S3?

[Gavin Carr]

On Tue, Apr 28, 2009 at 03:29:55PM -0400, Ed Blackman wrote:
> I'm exploring brackup as a replacement for my curent homebrew "monthly full 
> backups to DVD-R, incremental daily backups to CDRW" backup system.
>
> I plan to use the Amazon S3 target because I want to add reliable offsite 
> storage, and am willing to pay for it, but at the same time don't want to 
> pay to store bits that are already stored offsite in a reliable way: 
> unmodified files that my Linux distribution provides. (Ubuntu, in case it 
> matters) 
> In my current system, I do a lot of processing to remove unmodified distro 
> files (to save space, instead of money) from the list of files to back up 
> (and just dump the package and version of all distribution packages I have 
> installed).
>
> I don't see an easy way to hook that into brackup, though, and wanted to 
> see what other people do to backup files in /usr, /var, /etc, and other 
> system directories.

In general, you should really only need to backup /etc and perhaps certain
trees in /var. /usr should be only distro files (except perhaps /usr/local).
/etc should be relatively small too, so perhaps backing up distro files there 
isn't too bad size-wise.

I do something similar in spirit to what you're proposing though, using a 
couple of custom tools - extract and rpm-find-changes, both available here:
http://www.openfusion.com.au/labs/dist/. 

rpm-find-changes spits out a list of files within a tree that have changed from
their distro versions (using the rpm md5 checksums - I imagine it would be
straightforward to do something similar for apt). extract pulls a set of files
from a remote host into a local tree and run arbitrary scripts at various
points. I use it to pull /etc files that have changed from remote hosts and
version control them on my server, and then just point brackup at that entire
tree.

Ideas for your pot anyway.

Cheers,
Gavin


-- 
Gavin Carr
- http://www.openfusion.com.au - Linux, Perl, and Web Consulting
- http://www.openfusion.net    - Hackery, the Blog
* Fashion is a variable, but style is a constant - Programming Perl