Memcached suddenly dies

[Ivan Zolotukhin]

Hello,

We experience strange cases of Memcached death under moderate load of
50-80 reqs/sec. It happents approximately once per day in
unpredictable time. We use memcached for objects caching from a web
application (PHP) distributed on several web servers. Memcached
command string is:

memcached -d -p 11211 -u nobody -m 1024 -c 1024 -P
/var/run/memcached/memcached.pid

Main properties of the system:

CentOS 5.1
Kernel 2.6.18-53.1.13.el5PAE (32bit)
Memcached 1.2.5 built with ./configure && make && make install by GCC 4.1.2
Libevent 1.1a
2 x Xeon 5130
8GB RAM

We captured several straces and core dumps (generated by Memcached
specially built with debug) and can provide them to Memcached
developers privately if needed. Here is one example of the backtrace:

(gdb) bt
#0  0x004c4410 in __kernel_vsyscall ()
#1  0x00536ba0 in raise () from /lib/libc.so.6
#2  0x005384b1 in abort () from /lib/libc.so.6
#3  0x005301db in __assert_fail () from /lib/libc.so.6
#4  0x0804ee1d in do_slabs_alloc ()
#5  0x0804fd81 in do_item_alloc ()
#6  0x0804c9c9 in process_update_command ()
#7  0x0804d8b1 in try_read_command ()
#8  0x0804e174 in event_handler ()
#9  0x00653125 in event_base_loop () from /usr/lib/libevent-1.1a.so.1
#10 0x0804c35e in main ()
(gdb)

Problem is not related to hardware since we managed to reproduce it at
another server with the same configuration.

Any help is appreciated.

--
Regards,
 Ivan