Better server selection client code?
jeetum at gmail.com
Thu Oct 19 08:18:30 UTC 2006
On Wed, 2006-10-18 at 05:13 +0000, Christopher Baus wrote:
> > ----
> > "I've been toying with idea of using consistent hashes with memcached,
> > but I don't really have enough servers for it to be worth it.
> > http://www8.org/w8-papers/2a-webserver/caching/paper2.html
> > Leon"
> > ----
> That could be very cool. I read up on consistent hashes for load
> balancing crawlers. The promise of repartitioning the hash space at run
> time almost seems like a miracle. Memcached is probably the perfect place
> for implementing consistent hashes since the client and server is under
> the sysadmin's control, unlike the web.
But here you can have a problem of load distribution. According to the
algorithm, if 2 servers map to points very close on the unit circle, the
load distribution will be screwed badly.
possible solution - dont use hashing for mapping servers to the unit
circle, instead maintain a map, and pick the longest arc to place the
The paper also mentions in section 2.2:
For technical reasons detailed in , it is quite important to make a
small number of copies of each cache point--that is, to map several
copies of each cache to different ``random'' points on the unit circle.
This produces a more uniform distribution of URLs to caches.
This will also provide a good failover, as discussed here:
"The law of evolution is that the strongest survives!"
More information about the memcached