memcached performance on Mac OS 10.4

[Noah M. Daniels]

Ok, that makes more sense... so I disable both kqueue and NOPUSH.  
Does libevent need rebuilt with certain flags? Or will setting the  
EVENT_NOPOLL environment variable be sufficient?


On Mar 2, 2006, at 11:31 AM, Brad Fitzpatrick wrote:

> On Thu, 2 Mar 2006, Brad Fitzpatrick wrote:
>
>> If you rebuild libevent with kqueue support, and disable TCP_NOPUSH
>
> _without_ kqueue support, sorry.
>
>> support in memcached (which it tries to auto-detect), then  
>> memcached will
>> be fast on OS X, but you're limited in your number of concurrent
>> connections.  Or rather, it'll eat more CPU as you have more clients
>> connected because it has to poll()/select().
>>
>> The problem, as I understand it, is:
>>
>>    kqueue is auto-detected, but sucks on OS X.
>>
>>    NOPUSH is auto-detected, but sucks (on OS X and FreeBSD?) or its
>>      semantics diff from TCP_CORK on Linux.  not quite sure.
>>
>> In any case, a coworker of mine using memcached on OS X (for  
>> development,
>> not production), had success with the above.
>>
>> - Brad
>>
>>
>> On Sun, 15 Jan 2006, Peter Bengtson wrote:
>>
>>> I'm experiencing the previously reported 200 ms latency on each
>>> memcached query. As five queries per second is far, far worse than
>>> scrapping memcache altogether and running without memory caching, I
>>> was wondering if the problems on Mac OS 10.4 have been addressed  
>>> and/
>>> or worked around in any way since Gregory Block wrote the following
>>> to this list in March last year:
>>>
>>>> I had a bug open on Radar regarding memcached causing core dumps  
>>>> when
>>>> used in poll(), kevent(), or anything other than select.  I've been
>>>> informed by the team that there's a fix in place for Tiger, and  
>>>> that
>>>> they've tested under tiger and poll() without seeing the kernel
>>>> panics.
>>>>
>>>> So...
>>>>
>>>> Until that fix is in place, it's memcached + select() on mac os  
>>>> x, or
>>>> watch your kernel go tits up.
>>>
>>> Of course, libevent since v1.1 has a "workaround", but memcache when
>>> using libevent 1.1a and started in the following fashion:
>>>
>>> 	/usr/local/bin/memcached -k -uroot
>>>
>>> still reports:
>>>
>>> 	[warn] kq_init: detected broken kqueue; not using.: Bad file  
>>> descriptor
>>>
>>> and then proceeds to respond exactly five times a second.
>>>
>>> Also, James Robinson wrote, on Aug 30:
>>>
>>>> stores of payload between 14000 and 195846 bytes all happen on OS X
>>>> without delay. Apparently OS X's TCP_NOPUSH is just plain  
>>>> busted, as
>>>> seems to be the net wisdom garnered from google.
>>>
>>> So, I'm wondering what is happening here. Are the Mac OS 10.4
>>> implementations of kqueue and TCP_NOPUSH still broken? Are there any
>>> patches from Apple or any third-party solutions? And is the slow
>>> speed we're seeing the result of select() being inherently slow on
>>> Mac OS X? Is there another way of forcing memcached use poll()?
>>>
>>> 	/ Peter Bengtson
>>>
>>>
>>
>>