Memcached segfault on increment

[Janusz Dziemidowicz]

Hi,
I'm trying to resolve memcached segfault that hit me recently. I'm
using small instance of memcached (64MB) on Unix socket to collect
some statistics (counters). It was working great, but recently, some
new counters were added and memcached started to segfault after every
couple of minutes. Removing those counters makes it work again, so it
is rather strange. I've been able to catch Valgrind stack trace of the
crash:

==12825== Invalid write of size 1
==12825==    at 0x4A1C979: memset (mc_replace_strmem.c:479)
==12825==    by 0x40336D: do_add_delta (memcached.c:1535)
==12825==    by 0x403A36: process_arithmetic_command (memcached.c:1487)
==12825==    by 0x405EB4: try_read_command (memcached.c:1689)
==12825==    by 0x40642F: event_handler (memcached.c:2136)
==12825==    by 0x4B230E1: event_base_loop (in /usr/lib/libevent-1.1a.so.1.0.2)
==12825==    by 0x404862: main (memcached.c:3131)
==12825==  Address 0x559A000 is 0 bytes after a block of size 1,048,528 alloc'd
==12825==    at 0x4A1B858: malloc (vg_replace_malloc.c:149)
==12825==    by 0x406FEA: do_slabs_alloc (slabs.c:399)
==12825==    by 0x407A51: do_item_alloc (items.c:98)
==12825==    by 0x404DBB: process_update_command (memcached.c:1420)
==12825==    by 0x405BAB: try_read_command (memcached.c:1681)
==12825==    by 0x40642F: event_handler (memcached.c:2136)
==12825==    by 0x4B230E1: event_base_loop (in /usr/lib/libevent-1.1a.so.1.0.2)
==12825==    by 0x404862: main (memcached.c:3131)

I've been looking in the code myself, but function do_add_delta()
seems ok in the part given by Valgrind. I've been also trying to look
into memcached memory allocation functions, to search for anything
there, but that wasn't successful either.

I'm running Debian Etch AMD64 with hand compiled memcached 1.2.5 with
applied patch fixing another crash
(http://github.com/dustin/memcached/commit/6ec16c4). Process is run by
the command:
memcached -m 64 -s path -u nobody -M

Maybe someone could give me any advice on this one?

-- 
Janusz Dziemidowicz
Administrator
nasza-klasa.pl
phone: +48500298526
email: janusz.dziemidowicz at nasza-klasa.pl

Nasza Klasa Sp. z o.o., ul. Dembowskiego 57/5, 51-670 Wrocław
Sąd Rejonowy dla Wrocławia-Fabrycznej we Wrocławiu, VI Wydział
Gospodarczy Krajowego Rejestru Sądowego,
nr KRS:0000289629, NIP:898-21-22-104, REGON:020586020
Kapitał zakładowy: 67850 PLN