TCO and human hours for large deployments

dormando dormando at rydia.net
Thu Sep 20 05:48:26 UTC 2007


My first (*sniff*) MogileFS cluster was built on the 1.x series and was 
built and data migrating to it in four days. It had ten million files 
hosted by the fourth day of operation after that. It grew to many times 
that over two years. Note that we saved time because gaia had already 
been running on perlbal for a while, which was from another four day 
project a while earlier.

MogileFS is definitely a bit of work up front. The effort to get it in 
place at gaia was a combined effort with two members of the developer 
staff and myself. Then there were tweaks/fixes along the way. A day 
here, a day there.

That was before mogilefs started having options (lighttpd, apache, dav 
backends) and we didn't have a choice over the hardware it went on.

So if you want to research the best storage hosts, the best database 
setup etc, factor that all in.

Every couple months I'd spend a few days on it to fix some bugs. 
Occasionally something will break and you will have to focus on it, but 
it doesn't require constant babysitting.

That said, please use the mail archives, submit bugs, patches, 
questions, etc if you decide to further research mogilefs and dig more 
into it.

-Dormando

Lance Reed wrote:
> I am working out the Total Cost for a large (er, scalable...) mogilefs 
> deployment.  The numbers for hardware are relatively easy depending on 
> design.  However, like all opensource initiatives, calculating the time 
> / resources for administration by humans is tricky until you do it for 
> awhile.  Our company has plenty of experience using OpenSource tools for 
> the majority of our needs, with the proprietary solution thrown in where 
> applicable.  So I was wondering if anyone who is attempting to setup a 
> mogilefs storage setup, or who is using one now could shed some light on 
> the subject.  I figured initially, on a new deployment, about 60% of a 
> single Administrators time would be taken up to maintain a large 
> mogilefs cluster.  I figure design and implementation would take much 
> more than that (maybe 2 or 3 people to save time).  I am comparing this 
> to a couple proprietary solutions we have been looking at that are known 
> to scale well into multi TB or even PB setup that are basically Scalable 
> NAS solutions that use a "brick" mentality.  The theory being that you 
> pay upfront for stability or support if needed.  But I am wondering if 
> say 60% for the first year of learning how to maintain and grow a 
> production mogilefs is close or not.  Too high / low?
> 
> Any thoughts or real work data would be great.
> 
> Thanks all!!
> 
> 



More information about the mogilefs mailing list