Recovery procedure

[Brad Fitzpatrick]

You can reuse hostids if you want, it shouldn't hurt, but that might just
get confusing, so I wouldn't.

Leave the device row in the database, marked as dead.  I seem to recall
that's one of the mechanisms that kicks off the replication.

Mark Smith wrote a mogadm command for admining MogileFS recently... I
think it's only in our cvs, though.  You might want to check it out.

- Brad


On Tue, 8 Nov 2005, Brandon Ooi wrote:

> Hi Brad,
>
> Cool thanks. That works for me. Just for my information, do i leave the
> device in the table as dead or do I remove it after it's finished
> replicating (and if so, how do i know when it's finished replicating).
>
> if a host dies, should i never reuse host ids either?
>
> brandon
>
>
> Brad Fitzpatrick wrote:
>
> >Step 1:  NEVER REUSE DEVICE IDS.
> >
> >If you obey step 1, you can't mess anything up.  (But if you already did,
> >you can still recover...)
> >
> >Step 2:  modify the device table, marking the old device as dead.
> >
> >Step 3:  mark the host alive.
> >
> >Step 4:  make a new device (with a new device id), on the alive host
> >
> >
> >And then it'll just replicate for you.
> >
> >
> >
> >On Tue, 8 Nov 2005, Brandon Ooi wrote:
> >
> >
> >
> >>Hi,
> >>
> >>We suffered our first drive failure on mogile and I'm trying to figure
> >>out the aftermath.
> >>
> >>1) the primary drive on one of our storage nodes failed
> >>2) this caused both our mogilefsd daemons to hang requiring a restart...
> >>3) In SQL, i marked the host as 'down' so that requests would not go to
> >>that machine.
> >>4) I replaced the primary drive and brought the machine back up.
> >>
> >>There is a second drive on the machine that is still good and I'd like
> >>to use that. How do I mark the device as 'dead' and the host as 'alive'
> >>and have mogile rereplicate everything that was lost? Is there a
> >>standard way of doing this?
> >>
> >>brandon
> >>
> >>
> >>
> >>
> >
> >
> >
>
>