Overrepication Problem Revisited

Russ Garrett russ at last.fm
Sat Aug 18 12:32:17 UTC 2007


We do use MySQL replication, but at the moment there's only one tracker 
running on this system, so it shouldn't make any difference. There are 4 
nodes currently in the system, each with 6 devices on. At the moment 
we're batch importing small files (images) at a rate of about 2GB/hour.

The only other unusual thing is that we're using a separate instance of 
lighttpd on each node with a separate getport. I might try and change 
that to making mogstored start lighttpd.

Russ


stefan wrote:
> My two very general euro cents, but this has to be triggered by something
> in your local configuration - something that most people aren't doing, so
> I guess it would be cool to know a little bit more, about things like 
> mysql
> setup (cluster, replication), what are the actual classes and devcounts,
> what replication policy you're using (custom?), storage node server 
> (webdav
> or mogstored) and setup (devs, amount of free space), how you inject 
> files
> and when (ie. add a graph line with the nb. of files)...
>
> Cheers,
> Stefan
>
> On 8/18/07, *Russ Garrett* <russ at last.fm <mailto:russ at last.fm>> wrote:
>
>     I've just set up a brand new MogileFS cluster - the tracker is
>     2.17 and
>
>     passed all the tests - and it's definitely exhibiting an
>     overreplication
>     issue:
>
>     http://playground.audioscrobbler.com/russ/overreplication.png
>     <http://playground.audioscrobbler.com/russ/overreplication.png>
>
>     (overreplicated files are file.devcount > class.mindevcount,
>     underreplicated are where the reverse is true. Files to replicate
>     is the
>     size of file_to_replicate)
>
>     Any clues?
>
>     Cheers,
>
>     Russ Garrett
>     Last.fm <http://Last.fm> Ltd.
>     russ at last.fm <mailto:russ at last.fm>
>
>
>



More information about the mogilefs mailing list