ketama memcached java client

[Richard Jones]

Some quick and dirty testing at the time showed that 160 entries per server 
gave a decent distribution.. The java client just implements what the C 
library does.
I'm in the process of updating libketama to make this configurable and merging 
a patch that implements FNV hashing, which seems to be almost twice as fast 
as md5 (still need a java impl of FNV-1a)
Patch i was sent is here: svn://svn.audioscrobbler.net/misc/ketama/patches/
I'll post some better benchmarks and updates once i'm done cleaning up 
libketama.
RJ


On Monday 30 July 2007 01:11:26 Hanson Char wrote:
> Hi,
>
> I refer to the ketama SockIOPool
> (http://static.last.fm/ketama/ketama-0.1.1.tar.bz2) implementation for
> the use of consistent hashing in the memcached java client.  In the
> initialize method, if the weight of each server is 1, the "factor"
> would be 40 and therefore resulting in each server being added to the
> TreeMap 160 times (with code extracted below.)
>
> Curious, why 40 ?  Is it optimal ?  Why not configurable ?
>
> Hanson Char
>
> 			double factor =
> Math.floor(((double)(40*this.servers.length*thisWeight))/(double)this.total
>Weight);
>
> 			for(long j = 0; j < factor; j++) {
> 				byte[] d = md5.digest((servers[i]+"-"+j).getBytes());
> 				for(int h=0;h<4;h++) {
> 					Long k =
> 						  ((long)(d[3+h*4]&0xFF) << 24)
>
> 						| ((long)(d[2+h*4]&0xFF) << 16)
> 						| ((long)(d[1+h*4]&0xFF) << 8)
> 						| ((long)(d[0+h*4]&0xFF));
>
> 					buckets.put(k, servers[i]);
> 					log.debug( "++++ added " + servers[i] + " to server bucket" );
> 				}
> 			}



-- 
Richard Jones
Last.fm Ltd. | http://www.last.fm/
Office: +44 (0) 207 780 7080