Memcache proxy

[Timo Ewalds]

I've been using memcache extensively for the past year or so, and am now 
at the point that I want to use it for more than just to replace primary 
key lookups to a db. That generally means keeping track of a bunch of 
ids in some form, storing each item individually, and fetching them with 
a get_multi. With 30 memcache servers, and php (which doesn't support 
connection pooling and not great persistency), that could be rather slow 
if it's connecting to many servers. While I do generally try to keep 
related data on one server, that isn't always possible.

To solve this, we realized that using a memcache proxy would solve the 
problem. The proxy would keep one (are more needed or helpful?) 
connection to each memcached server, and one (persistant?) connection to 
each php process. It would then proxy commands to the right server based 
on a prefix to the key (ie keys would have to be of the form 
$hash-$key). The proxy could be run on each webserver so the php-proxy 
connections could all be local, though this isn't strictly necessary. 
This would greatly reduce the connection overhead, and the number of 
overall connections. For get_multi queries to multiple servers it should 
give great performance benefits when persistant connections and 
connection pooling aren't possible.

Has anyone worked on a system similar to this? Does anyone have any 
comments on how well this can/will work?

Timo