memcached performance on Mac OS 10.4

[Noah M. Daniels]

Thanks. This may be very helpful. I'm not worried about not  
supporting a ton of concurrent connections; this is an application  
that's going to be a self-contained Ruby on Rails app used by  
professional photographers. This is why it has to deploy on OS X  
rather than Linux. It's not going to be under terribly heavy load,  
but I was hoping to be able to use memcached to improve latency.

How do I build libevent with kqueue support?

Obviously, disabling TCP_NOPUSH in memcached is a simple #undef


On Mar 2, 2006, at 11:24 AM, Brad Fitzpatrick wrote:

> If you rebuild libevent with kqueue support, and disable TCP_NOPUSH
> support in memcached (which it tries to auto-detect), then  
> memcached will
> be fast on OS X, but you're limited in your number of concurrent
> connections.  Or rather, it'll eat more CPU as you have more clients
> connected because it has to poll()/select().
>
> The problem, as I understand it, is:
>
>    kqueue is auto-detected, but sucks on OS X.
>
>    NOPUSH is auto-detected, but sucks (on OS X and FreeBSD?) or its
>      semantics diff from TCP_CORK on Linux.  not quite sure.
>
> In any case, a coworker of mine using memcached on OS X (for  
> development,
> not production), had success with the above.
>
> - Brad
>
>
> On Sun, 15 Jan 2006, Peter Bengtson wrote:
>
>> I'm experiencing the previously reported 200 ms latency on each
>> memcached query. As five queries per second is far, far worse than
>> scrapping memcache altogether and running without memory caching, I
>> was wondering if the problems on Mac OS 10.4 have been addressed and/
>> or worked around in any way since Gregory Block wrote the following
>> to this list in March last year:
>>
>>> I had a bug open on Radar regarding memcached causing core dumps  
>>> when
>>> used in poll(), kevent(), or anything other than select.  I've been
>>> informed by the team that there's a fix in place for Tiger, and that
>>> they've tested under tiger and poll() without seeing the kernel
>>> panics.
>>>
>>> So...
>>>
>>> Until that fix is in place, it's memcached + select() on mac os  
>>> x, or
>>> watch your kernel go tits up.
>>
>> Of course, libevent since v1.1 has a "workaround", but memcache when
>> using libevent 1.1a and started in the following fashion:
>>
>> 	/usr/local/bin/memcached -k -uroot
>>
>> still reports:
>>
>> 	[warn] kq_init: detected broken kqueue; not using.: Bad file  
>> descriptor
>>
>> and then proceeds to respond exactly five times a second.
>>
>> Also, James Robinson wrote, on Aug 30:
>>
>>> stores of payload between 14000 and 195846 bytes all happen on OS X
>>> without delay. Apparently OS X's TCP_NOPUSH is just plain busted, as
>>> seems to be the net wisdom garnered from google.
>>
>> So, I'm wondering what is happening here. Are the Mac OS 10.4
>> implementations of kqueue and TCP_NOPUSH still broken? Are there any
>> patches from Apple or any third-party solutions? And is the slow
>> speed we're seeing the result of select() being inherently slow on
>> Mac OS X? Is there another way of forcing memcached use poll()?
>>
>> 	/ Peter Bengtson
>>
>>