first time user with out of memory question

[Steven Grimm]

Paul T wrote:
>  I don't follow your logic. 
>
>  Either :
>  
>  "it is so simple - it can not have bugs" or 
>  "I just fixed the last bug in that part of code" 
>
>  - it can not be *both* at the same time, right? 
>   

The benefit of using a very simple memory allocator in memcached is not 
that there is no possibility of bugs (of course any code can have bugs, 
even simple code), but that it's faster. A well-written fixed-chunk-size 
slab allocator takes less CPU time to manage its memory than a 
well-written heap manager like the system malloc() because there is less 
metadata to manipulate. It also never suffers from memory fragmentation 
because fragmentation is a side effect of variable-sized allocation.

On the other hand, memcached's slab allocator replaces fragmentation and 
greater CPU consumption with other behaviors that may or may not be 
worthwhile tradeoffs for a particular application: potentially large 
amounts of per-item overhead depending on how well your item sizes match 
the chunk sizes, and per-chunk-size LRU behavior rather than true 
cache-wide LRU. I attempted to reduce the first of those problems by 
allowing finer-grained chunk sizes, but it's impossible to avoid completely.

It bears mentioning that the slab allocator is a compile-time option in 
memcached. If those tradeoffs are not worthwhile for your application, 
you can recompile it to use the system malloc().

>  Unfortunately, memcached is an asynchronous
> application (based on libevent). For that kind of
> applications (timing/pattern/load/OS sesnitive) -- "it
> runs in my environment" does not mean that it would
> work for a different setup.
>   

I completely agree with you: it's always advisable to make sure a 
particular tool is the one you need for a particular job. "Your mileage 
may vary" is generally true of all software, asynchronous or otherwise. 
If someone tries out the Facebook version of memcached and finds that it 
doesn't work as well as the stock 1.1.12, they should absolutely use 
1.1.12. (And send a message to the mailing list detailing their 
experience in case it's a sign of a bug that needs fixing.)

-Steve