Memcache proxy

[Timo Ewalds]

mike wrote:

>On 11/1/05, Timo Ewalds <timo at tzc.com> wrote:
>  
>
>>Does php-mcache support connection pooling or similar? Having over 2000
>>php children across 36 servers, each with persistent connections to 30
>>memcache servers just seems rediculous when a proxy setup would reduce
>>that to only one connection from each physical webserver. At least with
>>pooling those 2000 children would share their connections. I know
>>memcache can support huge amounts of connections, but can it support
>>enough? can php support enough?
>>    
>>
>
>This I do not know. Perhaps it would be best to contact the guy who
>made it... maybe he has something in development, or a reason to not
>do things a specific way. Knowledge sharing is awesome++
>
>I suppose you could use the pmemcache() function and set a timeout
>that's appropriate, and just expect a lot of cross-network memcache
>traffic. It might not be able to hit the cache 100% of the time, but I
>think it would do it as much as resources allow.
>
>In theory you could use something as simple as datapipe, or perhaps
>LVS or some sort of load balancing mechanism to send all your requests
>through some sort of proxy idea... but I think that's starting to ruin
>the point of data being stored on only one memcache server - it HAS to
>connect to that server that is storing that specific key'ed data or
>it's pointless.
>
>(Unless you're talking about a "smarter" proxy designed specifically
>for memcache which has the awareness of key->memcache server
>mappings...
>
>  
>
Yes, we'd be writing a memcache specific proxy that understands which 
server to send the specific request to. For all intents and purposes, it 
could use a hashing algorythm on the keys to do that (which works as 
long as all clients use the proxies or the same hash algorythm), though 
we'd probably have it just use a prefix of the key for speed purposes. 
Using LVS or similar completely defeats the purpose of memcache as a 
distributed cache, and would lead to many problems of expired data being 
returned.

Timo