Confused on scaling memcached & persistant connections (w/ the PECL client)

[Timo Ewalds]

I considered writing one of these a while ago (prototype in php, real 
probably in c?). I posted about it to the list about 3 months ago. It 
would be a proxy type system that I'd run on each server (40+ right 
now). Each proxy would have one persistent connection to each memcached 
server, and one persistent connection to each php process on that server 
(I run 60 processes per server). That would essentially cut the 
connections to memcache by 60 (for me), and use only persistent 
connections. It shouldn't mean a big performance hit anywhere, though it 
does increase latency a bit. It would cut the total number of 
connections down from about 80,000 right now to 6000 or so.

I ended up not writing this, but just bumping up the connection limit to 
4000 (don't forget the ulimit), and using all persistent connections. 
With epoll, this works just fine.

Timo

Richard Thomas wrote:

> It wouldn't be to hard to make a daemon to do this, heck I could do it 
> in php but would would the +++ outway the ---
>
> The thought being if I get this right,
>
> Apache -> Daemon -> Nonpersistant, but local connection overhead is so 
> small it doesn't matter? Or would this be persistant?
>
> Daemon -> Memcache -> Persistent but pooled.
>
> This way you could get away with fewer connections overall to the 
> Memcache server?
>
>
>
> Richard Thomas - CEO
> Cyberlot Technologies Group Inc.
> 507.398.4124 - Voice
>
>
> Brian Moon wrote:
>
>>> Are there any plans for the PECL memcached client to pool the 
>>> persistent
>>> connections on the server in a more efficient manner?
>>
>>
>>
>> Each Apache process will have to have its own persistent connection. 
>> The only other option I could think of would be to have a daemon on 
>> the localhost that acted as a proxy.  I don't know of such a thing at 
>> this time.
>>
>>> Also, has anybody else noticed just how slow unserialize() is in PHP?
>>
>>
>>
>> Its not super fast, but its not much slower than creating the data 
>> every time in code.   At least, not when I last tested it.
>>
>>> Is there any way for the memcached PECL client to be extended to accept
>>> arrays/objects and do this faster
>>
>>
>>
>> To store an array or object, you have to make it a string or binary 
>> data somehow.  The only way to do that in PHP is serialize.
>>
>> Brian Moon
>> dealnews.com
>>
>
>