Cross-client memcached compatibility

[dormando]

Aaron Stone wrote:
> On Wed, 2008-01-30 at 13:34 +0300, Tomash Brechko wrote:
>> On Wed, Jan 30, 2008 at 10:04:28 +0000, Ciaran wrote:
>>> Even this (string, integer and floating point [byte-array also?] ), would be
>>> a better state-of-affairs for people who are working with the cache in a
>>> mixed platform (as I currently am!)
>> No doubt.  Actually my post where I gone into serialization details is
>> a bit off.  The whole matter is simply not relevant to this list.
>>
>> The only thing the client author should provide is serialization
>> hooks.  When user asks to serialize the data, the client calls
>> serialize hook, and rises SERIALIZED flag.  When some other client
>> sees SERIALIZED flag set, it calls deserialize hook before returning
>> the data to the user.  And it's up to the user to use consistent
>> serialization methods.
> 
> Or identify the method used in the data itself. It's quite a reasonable
> approach, IMHO.
> 
>> It works exactly this way for current (not all?) clients, the only
>> addition is that by default they set these hooks to serialization
>> library that comes with the language.  But this doesn't mean that they
>> are fixed on a particular serialization method that we have to
>> standardize.
>>
>> Thus, if one knows a library (or wrote one himself) that may serialize
>> across different platforms, and has bindings (APIs) to languages in
>> question, the user would simply hook it into his clients, that's all.
>> So the question is whether there's such a library that binds to
>> sufficient number of languages?  But that has nothing to do with the
>> clients themselves...
> 
> We do not need to define our own serialization system, nor do we need to
> Choose The One True Serialization Encoding To Rule Them All (TM)-- we
> just need to allow for enough information to identify what the data is.
> 
> Aaron
> 

I think that's what he's getting at.

Hmm. I wonder... Instead of just one serialization flag, we could have two.

The first, existing, one would mean "native language". Ruby marshal, 
perl Storable, PHP ... whatever the hell.
Second would be 'application serialized', which it'd then run a client 
callback as Tomash suggested. The client callback can use entirely its 
own scheme to serialize/deserialize the structures (run it through 
thrift, json, whatever).

Then all clients would "standardize" on some familiar API for this.
... and potentially provide example snippits that would use JSON.
It also means the current defaults will continue to work. In a bulk of 
cases you'd _want_ to just use the native method, since it's likely 
pretty fast and well enough tested.

Which is very close to what ya'll seem to be talking about :) Maybe?

-Dormando