Mogstored Tuning

[Brian Lynch]

I made the change to using lighttpd for reads across two of the storage
nodes and already see a reduction in the number of timeout errors.
There appears to be an unconfirmed improvement in the speed of our
application as well.  I'm letting it burn in a bit before rolling to the
remaining nodes, but things look good so far.  Thanks again for the
rapid response and great suggestions! 

- Brian

-----Original Message-----
From: dormando [mailto:dormando at rydia.net] 
Sent: Thursday, April 10, 2008 2:42 PM
To: Mark Smith
Cc: Brian Lynch; mogilefs at lists.danga.com
Subject: Re: Mogstored Tuning

You can get a little bit of an idea because a mogstored is essentially a

perlbal plugin. So the management interface works, and you can write a 
little extra code to track stats if you wanted.

Otherwise a general idea should be fine, or just purely a reqs/sec with 
an understanding of how many writes you're probably doing.

-Dormando

Mark Smith wrote:
>>   We are using mogstored for both reads and writes.  I wasn't aware
of
>>  the ability to split out reads, but I found the command in mogadm.
I'll
>>  give that a spin.  Thanks for the suggestion!
> 
> Definitely definitely do that, that's one of the first things we did
> when we actually started using MogileFS in a serious way on LJ.
> Apache2 works, lighttpd works, whatever can do GETs in a quick way
> will work!
> 
>>  Is there a way to gather statistics on the number of reads/writes
>>  passing through mogstored?
> 
> Not built in, but you should be able to determine this by just
> understanding your traffic.  Unless you are overwriting files
> constantly, or serving as a backup service where people don't get
> their data often, there's only a few writes per file.  One to insert,
> N to replicate up to mindevcount, and then many reads for serving.
> The files aren't touched again (unless you do something with fsck or
> rebalancing, but even then you're still heavily on the side of reads
> for most usage).
> 
>