[PATCH] utf8 flag support on perl lib

Tomash Brechko tomash.brechko at gmail.com
Sat Jan 12 20:59:18 UTC 2008


On Sat, Jan 12, 2008 at 12:25:57 -0800, Dustin Sallings wrote:
> 	I don't understand why you wouldn't want UTF-8 in common.

Because I think that such flag would be a mistake, and want to limit
the damage it will cause.  Imagine the application that reads a
key/value pair, and stores it to memcached (Brian may have one).  If
we decide to relay on UTF-8 flag when fetching data, then such app now
would have to ask, "Dear user, but what's the meaning of the data we
are uploading?  Is it binary, or a non-ASCII text?" (the data may come
from arbitrary file, you can't assume it will always be the text
encoded according to user's locale).  This information is obviously
not relevant for the uploading process itself, but we would be bound
to decide how the data would be retrieved later.  I find this very
ridiculous.


> 	What do people use flags for currently other than to indicate data  
> type?

At least in Perl, when you serialize data its type is encoded in the
new value, not separately in flags.  C::M uses only F_COMPRESSED and
F_STORABLE currently.


> 	I was under the impression that CRC32 hashing with  
> modulus bucketing was an LCD.

Even C::M does a bit more:

  ((crc32($key) >> 16) & 0x7fff) % number_of_servers

There's also what's called the Ketama consistent hashing, and there
you are free to choose any hash function for server names (C::M::F
uses the same CRC32).


-- 
   Tomash Brechko


More information about the memcached mailing list