GNU malloc vs. slab alloc

[timeless]

Brad Fitzpatrick wrote:

> Back in the day, memcached used GNU malloc.  After a week of usage in 
> production on LiveJournal, CPU usage would start growing, and then 
> growing really rapidly.  After day 8, it'd be pretty much unusable, as 
> memcached would be at 99.9% CPU usage the whole time.
>
> So we switched to Judy.... same thing.
>
> Then we wrote the slab allocator and haven't had problems in years. 
> (and literally, we've had memcached instances running for nearly a 
> year at a time....)
>
> So I'm a little skeptical.  I'm not sure your test is harsh enough.
>
> Torsten Foertsch wrote:
>
>> I wrote then a test program that stresses the server with a set 
>> operation with a random key followed by 2 get operations with keys 
>> randomly selected from the 200 recently generated keys. Between these 
>> operations the program sleeps for 0.15 sec. Thus, the server is 
>> stressed with approx. 2 set and 4 get operations per second.
>
...

>> The program ran for over a week against 3 servers
>
I'm kind'f with Brad here. How about 1,200 sets/second and 1,600 
gets/second for 30 days? If your CPU isn't stressed with this workload, 
keep doubling gets/sets until it is. If you get network bottleneck, 
reduce the size of the data you're storing and retrieving (being sure 
you still set/get large objects a couple times per second). Running with 
GigE should help reduce this possibility.

You could set/get randomised keys and data sizes across a memcached 
cluster with all your different configurations (slab vs. GNU malloc() 
vs. whatever special configuration you might think is best) so you could 
just run one test. Make sure each machine has to do a significant workload.

You could use the Cacti memcached graphing template (here: 
http://faemalia.net/mysqlUtils/) to watch the CPU usage as well as check 
the sets and gets/sec rate per server.

--
timeless