Cross-client memcached compatibility

Tomash Brechko tomash.brechko at gmail.com
Sun Feb 3 12:59:31 UTC 2008


On Sat, Feb 02, 2008 at 01:50:58 -0800, dormando wrote:
> For the first point; I don't understand why you'd want to arbitrarily
> limit serialization. If I write a gigantic web application which uses
> native serialization, and add a single experimental web service that
> pulls from a particular data stream, we should be afforded options.

I see your and Aaron's point, agree, with my proposal it's impossible
to use two different serialization methods.  You still may use native
and non-native formats inside a single hook, and encode the method you
used with, say, one byte header.  But this won't work with the old
clients that want to use native format while do not understand any
prefix byte.  Still: we are talking about the _native_ format.  I
perfectly understand why one would want to mix several clients for
_different_ languages, but why mix clients for _one_ language?

Anyway.  Note that having exactly _two_ different methods is limiting
as well, and it's a bad design to have those "special cases" for
disputable "problems".  Yet, imagine we agree to have two methods.
Current interface for, say, Perl, is

  $memd->set('key', $data);

If $data is a reference, a hook is called.  How would you specify
which serialization method to use?  With one hook, I can think of,
say,

  $memd->set('key', [$data_ref, $selected_method]);

>From module's POV it's still some opaque data structure (a reference
to a list), but I write the hook so that it expects two-elem list, and
acts according to the second element.  I would encode the used method
in the first byte of the serialized data.  Now, if it will be the
module that will have two different serialization methods, it will
have to decide _before_ calling any hooks, and such two-elem list will
become _part of the module interface_.

So your proposal is actually: let's change the interface of new
clients to allow the selection of serialization method (with old/new
switch, in addition to hooks).  What for?  To interact with other
clients for the _same_ language, those that don't have any hooks that
we could otherwise utilize.

And my proposal is: let's have just hooks.  If hook points to native
method, we are compatible with the old clients.  If hooks is custom,
we still may use native or whatever format, but may encode it in a way
incompatible with the old clients (use some header for instance).
This new flexibility is the advantage of new clients over old ones.
There's no need to make every new feature compatible with old clients.
The whole point is to do better, not to do the same.


-- 
   Tomash Brechko


More information about the memcached mailing list