Regarding Consistent Hashing....

[Richard Jones]

On Thursday 30 August 2007 06:22:53 Ajinkya Nahar wrote:
> Hi all,
>
> I want to know whether consistent hashing technique can be used in
> production environment.


At last.fm we are using libketama in production, which does consistent hashing 
for us. I would recommend starting off with consistent hashing, as it will 
save you major headaches when you add and remove servers.

Ketama is not built in to the php memcache client tho, which is a shame. I 
haven't yet found the time to test the branch in cvs for pecl memcache, which 
does more or less the same thing (but using crc32 as the hash). At the mo we 
use php_ketama in php to map keys to servers, which works but we don't get 
the benefits of pipelining and the new stuff in the pecl client.

I've tested fnv instead of md5 in libketama, which is significantly faster 
with a slightly better distribution. Next up is crc32 to see how that fairs.. 

fwiw:
svn://svn.audioscrobbler.net/misc/ketama
svn://svn.audioscrobbler.net/misc/ketama-fnv
contains the C lib, and a php, java and python implementation

RJ
>
> Is it still too young or is it of production quality?
>
> Also, what configuration needs to be done in the PHP client when using
> consistent hashing?
>
> Please let me know.
>
> Thanks in advance.
>
> Regards,
>
> Ajinkya



-- 
Richard Jones
Last.fm Ltd. | http://www.last.fm/
Office: +44 (0) 207 780 7080