Overrepication Problem Revisited
Russ Garrett
russ at last.fm
Sat Aug 18 12:32:17 UTC 2007
We do use MySQL replication, but at the moment there's only one tracker
running on this system, so it shouldn't make any difference. There are 4
nodes currently in the system, each with 6 devices on. At the moment
we're batch importing small files (images) at a rate of about 2GB/hour.
The only other unusual thing is that we're using a separate instance of
lighttpd on each node with a separate getport. I might try and change
that to making mogstored start lighttpd.
Russ
stefan wrote:
> My two very general euro cents, but this has to be triggered by something
> in your local configuration - something that most people aren't doing, so
> I guess it would be cool to know a little bit more, about things like
> mysql
> setup (cluster, replication), what are the actual classes and devcounts,
> what replication policy you're using (custom?), storage node server
> (webdav
> or mogstored) and setup (devs, amount of free space), how you inject
> files
> and when (ie. add a graph line with the nb. of files)...
>
> Cheers,
> Stefan
>
> On 8/18/07, *Russ Garrett* <russ at last.fm <mailto:russ at last.fm>> wrote:
>
> I've just set up a brand new MogileFS cluster - the tracker is
> 2.17 and
>
> passed all the tests - and it's definitely exhibiting an
> overreplication
> issue:
>
> http://playground.audioscrobbler.com/russ/overreplication.png
> <http://playground.audioscrobbler.com/russ/overreplication.png>
>
> (overreplicated files are file.devcount > class.mindevcount,
> underreplicated are where the reverse is true. Files to replicate
> is the
> size of file_to_replicate)
>
> Any clues?
>
> Cheers,
>
> Russ Garrett
> Last.fm <http://Last.fm> Ltd.
> russ at last.fm <mailto:russ at last.fm>
>
>
>
More information about the mogilefs
mailing list