mogstored dying: redux

[Andy Lo A Foe]

Hi,

In my experience WebDAV storage setup (lighttpd, nginx) are much
better at handling large chunks/files than mogstored. I use nginx in a
production environment with files ranging from a couple of bytes to a
gigabyte, no problem. In the pre-production tests I ran mogstored died
reliably with OOM's when handling 100MB+ files. Use mogstored only to
manage the usage stats on your storage nodes in that case.

Gr,
Andy

On Mon, May 19, 2008 at 3:25 AM, Greg Connor <gconnor at nekodojo.org> wrote:
>
> On May 18, 2008, at 5:59 PM, Ask Bjørn Hansen wrote:
>
>>
>> On May 18, 2008, at 17:54, Greg Connor wrote:
>>
>>>      Running.
>>>      Out of memory!
>>>      Out of memory!
>>
>>
>> Yikes.   64MB chunks shouldn't be that bad.  Are the storage nodes
>> otherwise loaded (high IO wait or some such).
>
>
> Nope, the storage nodes are doing nothing other than mogstored at this time.
>
>
>> Did you try using another HTTP server (lighttpd, nginx, apache, ...) for
>> the file transfers to the storage nodes?  I suspect most/many users use that
>> so mogstored doesn't get used that much in high traffic environments ...
>
> No I have not tried this.  Do you believe mogstored is pretty useless in a
> production environment?  If that's true and widely known, it's too bad the
> documents don't reflect this... Is there a document or list posting that
> explains what parts of mogilefs should be tuned (or outright replaced) for a
> high-traffic application?
>
> Are there documents stashed somewhere that I'm missing?  I looked at the
> "new" wiki (last updates about 5 and 10 months ago) and read everything
> available there, and I've read most of the man pages.  I keep finding stuff
> that I'm totally not getting.  I would welcome some advice or pointers on
> how to get apache set up to replace mogstored for file transfers...