concerning server list management, connection pooling
brad at danga.com
Thu Jan 18 00:39:48 UTC 2007
The code I wrote for consistent hashing can be used even without its
integration into the client library (if you use the consistent hashing to
build your servers => [...] list passed to the client constructor...).
But eventually it'll be integrated. You might take a stab at that if you
As for connection pooling: what exactly do you mean? The server does
epoll/kqueue... just keep a ton of connections open from every
mod_perl/php process on all machines.
On Fri, 12 Jan 2007, Ofer Nave wrote:
> I've known about memcached for a while, but only recently starting
> researching it for potential implementation within my company. I love the
> architecture, but there are two specific issues that concern me that I
> haven't heard have been solved yet:
> 1) modifying the server list (to add a new memcached server, for
> instance) destroys the key -> server mapping. the equivalent of a
> 2) connection pooling
> I realize both of these are not new issues to anyone on the list. I've
> tried skimming the archives to glean some insight from past discussions. I
> found a thread
> (http://lists.danga.com/pipermail/memcached/2006-October/002944.html) that
> referenced a research paper about consistent hashing. I also found a post
> by Brad
> (http://lists.danga.com/pipermail/memcached/2004-October/000804.html) that
> talks about the 'server list propagation' problem potentially being solved
> by running a set of trackers and using virtual buckets.
> Unfortunately, I can't find anything in the memcached docs themselves (at
> danga/memcached/, and in the docs of the PHP and CPAN modules) addressing
> these issues, or covering the use of trackers and virtual buckets, so I
> suspect that while clever individuals may have come up with various
> solutions, none of them have made it into the code.
> So, my questions are as follows:
> 1) What's the current consensus (or nearest to) on the list as to the
> best strategies for these two issues? (For the sake of context, assume an
> apache environment with mod_perl or mod_php).
> 2) Are there any working implementations of these solutions, either as
> part of the client libraries, or as additional tools (one of the posts
> mentioned creating a memcached admin tool)?
> 3) If not, can I help? I'm strongest with Perl, but can hack PHP if
> need be, and I'm likely to need both clients at my work. I'm also learning
> Python, so I'm up for a good limited-scope project.. I can write good code,
> but I have zero experience with memcached to date. So while I'm happy to
> help, I don't think I should be deciding what to implement, or how.
More information about the memcached