Verifying the InventoryDatabase

[William Wolf]

Hey Gavin,

I'm trying to use your script to verify the inventory database, but I am running into errors.  I am getting the following when I try to run ./brackup-verify-inventory -q --delete TARGET


Can't locate object method "count" via package "Brackup::Dict::SQLite" at ./brackup-verify-inventory line 111.


Did you have to modify lib/Brackup/Dict/SQLite.pm to add a count function there?  It also looks like it will be missing an each function, but I may be missing something here.  Thanks.


On Tue, Apr 28, 2009 at 8:43 PM, Gavin Carr <gavin at openfusion.com.au> wrote:
> Hi Ben,
>
> On Tue, Apr 28, 2009 at 12:19:08PM -0600, Ben Schumacher wrote:
>> Has anybody done any work on some sort of code to verify the
>> InventoryDatabase? I've noticed that I've had some issues with mine
>> being way out of sync with what's on S3. It turns out that some of this
>> has to do with some backups that failed during their runs, leading to
>> data orphaned from metafiles being sent to S3 and eventually getting
>> cleaned up by a later 'gc' operation. Part of this is the fault of a bad
>> script I have for automating my backups (it always gc'd, even if the
>> backup aborted -- now fixed).
>>
>> Anyway, rather than nuking my entire InventoryDatabase and possibly
>> forcing a reupload of many gigs of data to the TARGET, I'd like a tool
>> to do sanity checking of what's in the InventoryDatabase versus the
>> chunks already uploaded. I'd figure this would be a relatively simple
>> comparison of the 'value' column in the DB to the name of the objects in
>> the '-chunks' bucket on S3, but I figured I'd check if anybody has
>> already got some work start on this project.
>
> I wrote a quick brackup-verify-inventory utility a while ago, attached.
> There's a known bug with prune (I think?) not pruning the local directory
> properly, so I run this verify daily after backups, in --delete mode, since
> having orphaned chunks in your inventory is a really nasty failure mode.
>
> Cheers,
> Gavin
>
>



-- 

William Wolf

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 270 bytes
Desc: OpenPGP digital signature
Url : http://lists.danga.com/pipermail/brackup/attachments/20090505/c1babbb0/signature.pgp