Mogstored Tuning

[Brian Lynch]

Dormando,

  We are using mogstored for both reads and writes.  I wasn't aware of
the ability to split out reads, but I found the command in mogadm.  I'll
give that a spin.  Thanks for the suggestion!

Is there a way to gather statistics on the number of reads/writes
passing through mogstored? 

Best,
Brian

-----Original Message-----
From: dormando [mailto:dormando at rydia.net] 
Sent: Thursday, April 10, 2008 2:21 PM
To: Brian Lynch
Cc: mogilefs at lists.danga.com
Subject: Re: Mogstored Tuning

Are you using a separate read service or just the mogstored for reads 
and writes?

mogstored's ability to serve files without timing some out occasionally 
is a little suspect at the moment. A common practice is to fire up 
lighttpd or apache on a separate port and set the 'http_get_port' 
(you'll have to look up the actual mogadm command?) to point to that 
other web instance. Then the mogstored only handles writes and 
statistics for mogilefs, which it should be fine at doing.

-Dormando

Brian Lynch wrote:
> Hi,
> 
>  
> 
>   Does anyone have any experience tuning a larger installation of 
> MogileFS?  We have approximately 70 million files distributed among 5 
> large storage nodes (46 devices each; 2 X Dual Core AMD; 16G memory).
We 
> started experiencing a number of connection timeouts from the systems 
> hosting the tracker (mogilefsd) to the backend storage nodes 
> (mogstored).     We've made the following tweaks to reduce the
failures, 
> but haven't been able to determine why mogstored is timing out.  If 
> anyone has any ideas on tuning mogstored/perlbal or tweaking the
overall 
> installation, that would be extremely helpful.
> 
>  
> 
> Thanks,
> 
> Brian
> 
>  
> 
>  
> 
> TIMEOUT CHANGES
> 
>  
> 
>  
> 
> /usr/lib/perl5/site_perl/5.8.8/MogileFS/Worker/Replicate.pm
> 
>  
> 
> 71:  Changed watchdog_timeout to 60
> 
>  
> 
> 796: Changed Timeout to 50
> 
> 822: Changed timeout to 50
> 
>  
> 
>  
> 
> /usr/lib/perl5/site_perl/5.8.8/MogileFS/Worker/Monitor.pm
> 
>  
> 
> 88: Changed connection timeout to 10
> 
>  
> 
>  
>