Memcached segfault on increment

Hugo Hallqvist hugo at dokad.se
Wed Jun 18 12:36:15 UTC 2008


I don't know if this applies to your case or not, but when we debuged
our problems (which were fixed by the patch you linked to), we got
very different result from the core-file when running through
valgrind, and when not running through valgrind. Try running it
without valgrind and check the core-file afterwards with gdb (don't
forget "ulimit -c unlimited" prior to running memcache).

//Hugo

2008/6/18 Janusz Dziemidowicz <janusz.dziemidowicz at nasza-klasa.pl>:
> Hi,
> I'm trying to resolve memcached segfault that hit me recently. I'm
> using small instance of memcached (64MB) on Unix socket to collect
> some statistics (counters). It was working great, but recently, some
> new counters were added and memcached started to segfault after every
> couple of minutes. Removing those counters makes it work again, so it
> is rather strange. I've been able to catch Valgrind stack trace of the
> crash:
>
> ==12825== Invalid write of size 1
> ==12825==    at 0x4A1C979: memset (mc_replace_strmem.c:479)
> ==12825==    by 0x40336D: do_add_delta (memcached.c:1535)
> ==12825==    by 0x403A36: process_arithmetic_command (memcached.c:1487)
> ==12825==    by 0x405EB4: try_read_command (memcached.c:1689)
> ==12825==    by 0x40642F: event_handler (memcached.c:2136)
> ==12825==    by 0x4B230E1: event_base_loop (in /usr/lib/libevent-1.1a.so.1.0.2)
> ==12825==    by 0x404862: main (memcached.c:3131)
> ==12825==  Address 0x559A000 is 0 bytes after a block of size 1,048,528 alloc'd
> ==12825==    at 0x4A1B858: malloc (vg_replace_malloc.c:149)
> ==12825==    by 0x406FEA: do_slabs_alloc (slabs.c:399)
> ==12825==    by 0x407A51: do_item_alloc (items.c:98)
> ==12825==    by 0x404DBB: process_update_command (memcached.c:1420)
> ==12825==    by 0x405BAB: try_read_command (memcached.c:1681)
> ==12825==    by 0x40642F: event_handler (memcached.c:2136)
> ==12825==    by 0x4B230E1: event_base_loop (in /usr/lib/libevent-1.1a.so.1.0.2)
> ==12825==    by 0x404862: main (memcached.c:3131)
>
> I've been looking in the code myself, but function do_add_delta()
> seems ok in the part given by Valgrind. I've been also trying to look
> into memcached memory allocation functions, to search for anything
> there, but that wasn't successful either.
>
> I'm running Debian Etch AMD64 with hand compiled memcached 1.2.5 with
> applied patch fixing another crash
> (http://github.com/dustin/memcached/commit/6ec16c4). Process is run by
> the command:
> memcached -m 64 -s path -u nobody -M
>
> Maybe someone could give me any advice on this one?


-- 
Med vänlig hälsning,
Hugo Hallqvist
Dokad Software AB
www.dokad.se


More information about the memcached mailing list