Mogile Deployment Layout: More Hosts or More Disks.
dormando at rydia.net
Wed Sep 19 03:46:57 UTC 2007
> And this is where the issue comes. Right now, the usage patterns of
> our system will change. We envision this as i said as more of an
> archival system with light file serving, although this may well
> Our growth in images is about 7GB/day with text, uncompressed, being
> around 1-2GB. But we do expect this to double every 3-6 months.
> This of course in addition to our main data load, which will be about
> 3TB of images, and less than a TB of text ( since MogileFS will be
> doing compression)
If it changes, buy more ram and start adding hosts with fewer disks :)
If you can't really predict how your access patterns are going to be at
all, plan for change. Otherwise there's no real point in thinking about it.
> Figured that was one of the ways to alleviate this(cpu), by spreading
> them across, and running as many as possible.
There're some issues with DB load and running too many replicators or
whatnot. If you have a clever way to manage all of the running jobs (or
patch mogilefs to manage that centrally) then having tons of them
shouldn't be a huge deal I guess.
What I refer to with the trackers is that its CPU patterns can be
bursty. Just be aware of it I guess.
> The master-master solution sounds actually pretty interesting, and its
> something i'd probably want to implement if it works well enough.
> Would there be huge contention issues? do you just create a vip and
> round robing between the masters?
As the others said, master:master just makes it easy to switch back and
forth for outages and maintenance. It shouldn't be used as a load
balancing measure. The easiest way to prevent high load is to cache
active paths in memcached as far up in your application as possible, so
you only have to talk to the trackers for cache misses and writes.
More information about the mogilefs