Analysis of the over-replication issue
russ at last.fm
Wed Jul 25 00:08:08 UTC 2007
Nope, we're very careful to keep our MySQL slave on read_only mode, as
well as all the worker counts at 0 on the slave - doing otherwise is
usually quite dangerous.
Brandon Ooi wrote:
> Perhaps it's because you have the replicate workers running on more
> than one mogilefsd instance? Maybe they will run into each other and
> replicate more than once?
> We have it set up so only one machine runs the replicate, delete workers.
> I have no idea how that all works but that's how we have it set up.
> Russell Garrett wrote:
>> info at bouncetek.com wrote:
>>> I'm also experiencing excessive replication on a large scale. I've
>>> got a mogilefs setup running on 5
>>> hosts with about 49 devices. There are 4 file classes. 3 of them
>>> with a mindevcount of 3 and 1 with
>>> a mindevcount of 2. However a huge number of files get replicated to
>>> each and every disk.
>> For what it's worth, we're not experiencing this issue any more. We
>> added a ton more storage, updated mogilefsd and made sure all the
>> tests passed. Now it works fine (although we have a lot of
>> over-replicated files which I've had to write a script to sort out).
>> It's a shame we couldn't locate the cause, though.
>> Russ Garrett
>> Last.fm Ltd.
>> russ at last.fm
More information about the mogilefs