Client interoperability, server selection.

[Garrett, Russell]

I've become more interested in this topic recently because one of our
datacenter providers has decided, in their infinite wisdom, to turn the
power off twice in April. Since that's the equivalent of a complete
rehash on our end, and we might as well try and throw our weight behind
a standardised implementation.

> While developing Cache::Memcached::Fast we considered using libketama,
> but it does too many things (I/O, shared memory, synchronization),
> plus it uses MD5 hash, which I think is an overkill: there's no
> requirement for server name hash to be irreversible, plus MD5 is 16
> bytes when only 4 bytes are needed.  It's not a question of
> performance of course, as server additions/deletions are rare, but
> more a matter of minimalistic approach.

When we designed ketama we found that CRC32 had poor distribution,
although we may have missed something. Also, ketama does use all 16
bytes of the MD5 result by generating multiple continuum points per MD5
operation.

What is the distribution of C::M::F like (we have a test in the Ketama
distribution to test this)? If the distribution is fine, I think
Ketama-with-CRC32 is probably the best way forward.

--
Russ Garrett
Last.fm Ltd.
russ at last.fm