Intro / Busy loop and blocking threads on 1.2.5

Michael D'Agosta mdagosta at stumbleupon.com
Wed May 7 21:37:16 UTC 2008


Dormando, thanks for the info :)

Right now I've isolated the types of data I'm putting into the memcache. In one 
instance are counters only, and in another instance is metadata (like arrays of 
keys in the first instance). Long way of saying I'm isolating the problem in our 
application.

Haven't tried the patch yet (waiting on my app changes to crash :) ), and am 
beginning to read through assoc.c, naively wondering if the lookahead patch will 
work as expected. If I understand right, both the lookahead and it pointers are 
moving targets, and would only detect local loops 2 deep. Wouldn't you want to 
save an orig for comparing to lookahead?

Do you have a real-time communication mechanism, like IRC? This kind of thing 
might be easier done there.

You know, I haven't tried the 64bit compile-time option, and I'm trying to use a 
large amount of memory (8GB). What is the effect of compiling with the 64bit 
option?

Thanks,
-Michael

dormando wrote:
> Sorry to hear :\
> 
> Well, hold tight for now.
> 
> Info for the list: This is likely another case of the assoc_find bug.
> We're putting all bughunting resources into tracking it down right now.
> Hopefully we will have a fix soon, and I will be sending more info to
> the list to help out as soon as I can.
> 
> The bug's in assoc.c somewhere - Something causes the linked lists to
> have a loop.
> 
> Michael; Check the archives a few days back for a post from trond norbye
> with a patch for detecting and bombing out on such loops. If you try
> that out it could help us narrow things down.
> 
> We've otherwise been completely unable to reproduce the bug outside of
> the wild.
> 
> Thanks,
> -Dormando
> 
> Michael D'Agosta wrote:
>> The latter - both pegging the CPU and not accepting new connections.
>> This only happens every couple of days, but I can notice any other
>> symptoms that are relevant...
>>
>> -MD
>>
>> dormando wrote:
>>> Hey,
>>>
>>> Is memcached simply not accepting new connections and idling, or is it
>>> pegging the CPU and not accepting new connections?
>>>
>>> -Dormando
>>>
>>> Michael D'Agosta wrote:
>>>> Hello,
>>>>
>>>> New to the list - hi people. I was reading a thread from April:
>>>>   http://lists.danga.com/pipermail/memcached/2008-April/006683.html
>>>>
>>>> It seems that during busy times, one of our memcached instances will
>>>> stop accepting new connections; when I run telnet host port, I don't get
>>>> a prompt. It's MC 1.2.5 on CentOS 4.6 w/ libevent 1.1a. We didn't
>>>> compile in threading before, so I'm giving that a test drive as I type.
>>>>
>>>> Did anyone discover solutions or workarounds to the problem?
>>>>
>>>> Thanks in advance,
>>>>
>>>> Mike
> 


More information about the memcached mailing list