Replication Craziness

Nathan Schmidt nathan at pbwiki.com
Wed Jun 20 20:48:03 UTC 2007


During development and bring-up I used the stats command (wrapped in  
our own admin gui) but it quickly became apparent that running the  
full stats against a real instance was a good way to completely bone  
other activity on the dbs. Basically thrashed for a minute or two,  
blocking all writes.

If there were a way to run and very low priority or 'ok to be  
approximate' query against the primary db like Teradata allows this  
might be tractable but otherwise it's murder. I've removed the full  
stats command from our interface -- best not risk it being triggered  
accidentally.

For backup, metrics and general stats we have a single slave db  
chained off of our secondary master, so the mogile dbs look like this:

[Master primary machine] <----> [Master secondary machine]  -->  
[Slave machine]

The trackers point to a virtual IP juggled between the two masters  
via heartbeat. Works for us.

-nathan


On Jun 20, 2007, at 1:38 PM, Brad Fitzpatrick wrote:

> Basically the stats aren't computed in constant time, many of them  
> having
> to scan a ton of rows to be calculated.  And then the client has a  
> timeout
> of how long it'll wait for the server.... I guess the fix is to  
> increase
> the client's timeout for just that one command ("get_stats" or  
> whatever).
>
> - Brad
>
>
> On Wed, 20 Jun 2007, Erik Osterman wrote:
>
>> We have the same probably with stats....
>>
>> # mogadm stats
>> Fetching statistics...
>> Can't fetch stats
>>
>> mock wrote:
>>> On Tue, Jun 19, 2007 at 06:24:44PM +0100, Russell Garrett wrote:
>>>
>>>> Mogadm stats has recently stopped working for some reason: I  
>>>> just get
>>>> "Can't fetch stats".
>>>>
>>>>
>>>
>>> I noticed that stats stopped working after I hit around half a  
>>> million files.
>>> It recentently started working again after I dropped down below  
>>> about 300000.
>>> Probably something is broken there...
>>>
>>> mock
>>>
>>
>>



More information about the mogilefs mailing list