GNU malloc vs. slab alloc

[Torsten Foertsch]

On Friday 28 April 2006 01:53, you wrote:
> Back in the day, memcached used GNU malloc.  After a week of usage in
> production on LiveJournal, CPU usage would start growing, and then
> growing really rapidly.  After day 8, it'd be pretty much unusable, as
> memcached would be at 99.9% CPU usage the whole time.
>
> So we switched to Judy.... same thing.
>
> Then we wrote the slab allocator and haven't had problems in years.
> (and literally, we've had memcached instances running for nearly a year
> at a time....)

Yes, I know, I have read doc/memory_management.txt. That's why it is a command 
line option.

> So I'm a little skeptical.  I'm not sure your test is harsh enough.

The slab allocator is certainly best choice when LRU feature is used heavily. 
In this case the cache is almost everytime full. The allocation process then 
looks for the least recent block that is bigger than the currently needed one 
and frees it only to allocate it again with a slightly different size. This 
leads of course to a fragmentation problem.

With active expiry this changes. Now expired items are early removed from the 
cache. Hence, there are always several holes to choose from. This means less 
fragmentation.

Torsten
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.danga.com/pipermail/memcached/attachments/20060428/ca434f8b/attachment.pgp