[PATCH] utf8 flag support on perl lib
Dustin Sallings
dustin at spy.net
Sat Jan 12 21:41:52 UTC 2008
On Jan 12, 2008, at 12:59, Tomash Brechko wrote:
> On Sat, Jan 12, 2008 at 12:25:57 -0800, Dustin Sallings wrote:
>> I don't understand why you wouldn't want UTF-8 in common.
>
> Because I think that such flag would be a mistake, and want to limit
> the damage it will cause. Imagine the application that reads a
> key/value pair, and stores it to memcached (Brian may have one). If
> we decide to relay on UTF-8 flag when fetching data, then such app now
> would have to ask, "Dear user, but what's the meaning of the data we
> are uploading? Is it binary, or a non-ASCII text?" (the data may come
> from arbitrary file, you can't assume it will always be the text
> encoded according to user's locale). This information is obviously
> not relevant for the uploading process itself, but we would be bound
> to decide how the data would be retrieved later. I find this very
> ridiculous.
I may not be getting your point, but are you saying that the UTF-8
flag is bad because someone might want to store data that isn't UTF-8
and you need to be able to tell the difference? Isn't that the
purpose of the flag.
I have separate flags for byte array and string. I don't think
that's that unusual.
>> I was under the impression that CRC32 hashing with
>> modulus bucketing was an LCD.
>
> Even C::M does a bit more:
>
> ((crc32($key) >> 16) & 0x7fff) % number_of_servers
>
> There's also what's called the Ketama consistent hashing, and there
> you are free to choose any hash function for server names (C::M::F
> uses the same CRC32).
Yes, I suppose both of these models and have tests that verify that I
get the same results as the perl code above and the ketama C code.
--
Dustin Sallings
More information about the memcached
mailing list