Memcached crashing on FreeBSD

[Jason Coene]

Thanks guys.

I had the daemon running as a user who couldn't write to the directory it
was in, so I don't have a core to work with.  That should be resolved now;
it takes around 3-4 days for the crash to occur so I'm waiting until then.
Core file size is unlimited.

Also, I just found this in screen, the results of the crash inside of the
restart script:

Command -u pgsql -p 11212 -m 512 -c 1024
------------------------
memcached in malloc(): error: allocation failed
Abort trap

The same output above occurs with every crash.  I'll send a backtrace when
it crashes again.

Thanks,

Jason

> -----Original Message-----
> From: Evan Martin [mailto:evan.martin at gmail.com]
> Sent: Saturday, April 09, 2005 12:12 AM
> To: Brad Fitzpatrick
> Cc: Jason Coene; memcached at lists.danga.com
> Subject: Re: Memcached crashing on FreeBSD
> 
> You can get backtraces from core files, too.
> 
> ulimit -c unlimited
> then run memcached until it crashes, then
> gdb /path/to/memcached /path/to/core/file
> then run the "bt" command and send it to the list.
> 
> On Apr 8, 2005 1:36 PM, Brad Fitzpatrick <brad at danga.com> wrote:
> > Could you run a debug build and run gdb in screen, then run your runner
> > script in another window, waiting to start it up once the other one
> > crashes?
> >
> > I guess it wouldn't start up until the crashed one (still alive in gdb)
> > closes its listening socket on port 11211.   Is there a gdb command to
> > make a program close all its sockets on crash?
> >
> > Or in gdb just type "run <args>" and then once it starts, immediately
> type
> > "bt<enter>" so it'll backgrace on crash.  In another window, run a
> script
> > to every second check if memcached isn't responding, and if not, kill
> gdb
> > (it already did its backtrace presumably) and then run your normal
> runner.
> >
> > I'm down for other suggestions, though.
> >
> >
> > On Fri, 8 Apr 2005, Jason Coene wrote:
> >
> > > Hello,
> > >
> > > I'm using memcached in a fairly high traffic environment (but then
> again,
> > > who among us aren't!).  I have the memcached server running on FreeBSD
> > > 5.2-RELEASE and 5.3-RELEASE machines, and every day or two the service
> > > suddenly crashes (on both machines).  There are no hardware issues
> with
> > > either machine.
> > >
> > > >From dmesg:
> > >
> > > pid 2408 (memcached), uid 70: exited on signal 6
> > > pid 4872 (memcached), uid 70: exited on signal 6
> > > pid 9778 (memcached), uid 70: exited on signal 6
> > > pid 15861 (memcached), uid 70: exited on signal 6
> > > pid 24009 (memcached), uid 70: exited on signal 6
> > > pid 28084 (memcached), uid 70: exited on signal 6
> > > pid 33100 (memcached), uid 70: exited on signal 6
> > > pid 45761 (memcached), uid 70: exited on signal 6
> > > pid 58788 (memcached), uid 70: exited on signal 6
> > >
> > > Libevent 0.9 built with:
> > > ./configure --prefix=/usr/local/
> > >
> > > Memcached 1.1.11 built with:
> > > ./configure --prefix=/usr/local/ --with-libevent=/usr/local/
> > >
> > > Memcached is run inside of a restart script (this has been happening
> for a
> > > while, just tracking it down now) with commands:
> > >
> > > screen -A -m -d -S "Memcached 11211" /usr/local/bin/memcachedrun -u
> > > memcached -p 11211 -m 1024 -c 1024
> > >
> > > "memcachedrun" is just an infinite loop calling
> /usr/local/bin/memcached $*
> > >
> > > I've searched but couldn't find any known issues, has anyone had
> similar
> > > problems?  Is there something I can do to debug the issue?
> > >
> > > Thanks,
> > >
> > > Jason
> > >
> > >
> >