Questions about MogileFS in a production environment
otyaglo at friendster.com
Tue Dec 25 23:17:10 UTC 2007
We have quite large mogile storage setup running in production.
Our current mogile cluster has 44T of storage where 19T used.
Total number of stored files is ~330 millions (x3 copies ~= 1 billion
We customized mogile cluster next way:
- 8 federated db boxes with backup mysql replication
- 2 trackers per db group (16 total)
- 23 storage nodes, each has 8 250-320G drives (one for each group)
Our performance is
- 50 writes per second per whole cluster (tested up to 500 wps)
- 120+ reads per second per storage node (~ 3000 reads per second on a
- 80% full device recovery (re-replication) usually takes 30 hours with
no affect to production performance.
We developed custom web gui tool and init.d scripts to manage multiple
(8) instances of storage daemons (mogstored) on a given storage node.
GUI tool also comes handy with graphical display of usage pattern and
device/storage node availability.
In addition, we designed custom nagios checks that trigger notifications
about tracker/storage node availability along with single drive (device)
I hope you find this helpful.
From: mogilefs-bounces at lists.danga.com
[mailto:mogilefs-bounces at lists.danga.com] On Behalf Of Justin Huff
Sent: Thursday, December 13, 2007 9:42 AM
To: Clinton Goudie-Nice
Subject: Re: Questions about MogileFS in a production environment
I'm not sure we'd count as 'large scale' -- but we use it at Picnik.
We have about 5.7TB, using about 2.7TB. That's about 6.6million files
with a replcount of 2.
Clinton Goudie-Nice wrote:
> Dormando was kind enough to point me at some info in the mailing list
> from a few weeks ago where people talked about the scale of the
> stores they're currently using.
> Could anyone mention some names of some reasonably large scale web
> sites/services which I might recognize that are using MogileFS?
> Also, I've read a bit about trouble serving files greater than 100mb.
> there some limit on file size that I should be aware of?
> Thanks again!
> Twelve Horses <http://web.twelvehorses.com/> Mobile
> <http://web.twelvehorses.com/info_center/whitepaper/> Social
> <http://web.twelvehorses.com/solutions/website_design/> Email
> *Clinton Goudie-Nice*
> Architect / Senior Software Engineer
> cgoudie at twelvehorses.com <mailto:cgoudie at twelvehorses.com>
> Phone: +1.801.571.2665 ext 3264
> Mobile: +1.801.915.0629
> Fax: +1.801.571.2669
> LinkedIn: http://www.linkedin.com/in/cgoudie
> *Twelve Horses*
> 13961 Minuteman Drive
> Suite 125
> Draper, UT 84020
> *www.twelvehorses.com <http://www.twelvehorses.com>*
> On Wed, 2007-12-12 at 16:06 -0700, Clinton Goudie-Nice wrote:
>> Is anyone on the list using mogile in a large scale production
>> I'm considering running MogileFS in a production environment; sort of
>> a giant disk in the sky, for
>> an assortment of Internet apps. It will potentially be serving up
>> terabytes of data, mostly in small image files, probably behind a
>> squid caching proxy.
>> Is there anything people on the list could comment on? How stable is
>> it, how big are your various production stores, etc...
More information about the mogilefs