Alternative Binary Protocol idea for memcached.

Fri Feb 22 02:40:05 UTC 2008

This list is the best.

Then there's #memcached on irc.freenode.net

Then there's the wiki linked from danga.com/memcached/

Development and user assistance both happen on this list. It's
relatively low volume. I don't know of any web forums...

joe schmo wrote:
> 
> Does anyone know of a good site to post questions on regarding problems
> with memcahced?
>  
> 
>  
> 
>     ------------------------------------------------------------------------
>     Date: Thu, 21 Feb 2008 02:42:10 +0000
>     From: webb.clint at gmail.com
>     To: memcached at lists.danga.com
>     Subject: Re: Alternative Binary Protocol idea for memcached.
> 
>     Alright, I've started looking at the code. 
> 
>     Would anyone object if I first started working on abstracting the
>     protocol handling routines so that they are handled by callback
>     instead of the current if/else setup?
> 
>     Then I was going to put all the protocol stuff in prot_ascii.h|c and
>     prot_binary.h|c respectively.
> 
>     This way we could potentially have hundreds of protocols handled
>     without having a lengthy if/else comparison.
> 
>     On Wed, Feb 20, 2008 at 10:06 AM, Clint Webb <webb.clint at gmail.com
>     <mailto:webb.clint at gmail.com>> wrote:
> 
>         I wasn't expecting such a quick reply.  Good point about
>         allowing multiple protocols.  I might pull out some of my old
>         code and see how easy it is to drop in.
> 
>         I thought I'd give a little background on myself and this
>         protocol style. I used to work in the controls and automation
>         industry.  If you've ever checked your luggage into an airport,
>         or sent anything thru USPS, UPS or FedEX, or bought anything
>         online from amazon, b&n and other large online stores, or from
>         Walmart, kmart, target, etc... then your product has likely had
>         some experience with my coding somewhere along the line.
> 
>         I developed this protocol for a tiny little side-project for one
>         of the above mentioned companies.  It was basically a connector
>         that took information from a large number of different systems
>         and passed it to another.  The requirements changed a lot, so I
>         developed something that could be fast, but had to be flexible. 
>         If I added a feature to the server, I didn't want to be forced
>         to update all the clients as well.   Also, some of the clients
>         were tiny little embedded controllers, so it had to be pretty
>         simple too.
> 
>         This solution was VERY fast, as all the commands are a single
>         byte and could be easily mapped to an array of callback
>         routines.   This protocol also had to run on a real-time system
>         also, so we had to ensure that all operations preformed in a
>         predictable fashion.
> 
>         I seperated the commands by their parameter type.  0-31 had no
>         parameter.  32-63 had a single byte parameter.  64-95 had a
>         32-bit parameter.  96-127 had a single-byte paramter which was
>         the length of a stream of data to follow (short-string). 
>         128-159 had a 32-bit integer that was the length of a stream of
>         data to follow.   This was our 5 different parameter types.  A
>         command could only have one parameter.
> 
>         This way, the IO handling code could retrieve all the necessary
>         data and then pass it off to a callback routine for that command.
> 
>         Each socket connection would have a set of buffers set aside for
>         the incoming data.  In this case we would want a buffer set
>         aside to hold the key and value data.
> 
>         To speed up processing and ensure that the minimum data set has
>         been provided, we used a 32-bit (or was it 64-bit?) word as
>         flags.   Each operation would set or clear a flag(s).   So when
>         a GO command is received, it can quickly determine what 'action'
>         needs to take place,  and which 'parameters' have been provided.
> 
>         If we ran out of room having to handle more than 256 commands,
>         we would use a command of 0xFF which would expect that the next
>         byte would be another command (from a set different to the
>         first).  I never actually implemented it though.  The most
>         commands I ever used was about 100 or so.
> 
>         I cant imagine that a variable-length structured protocol could
>         be much faster than that.  Still the emphasis of this protocol
>         is not so much on speed, but on flexibility to add functionality
>         to the protocol (by adding commands) without breaking existing
>         clients (and without having to handle multiple versions of the
>         protocol).
> 
>         The 'noreply' stuff that I have seen around the list could
>         probably benefit from this protocol.  I haven't looked close
>         enough at the CAS stuff either, but I suspect that would be easy
>         to implement too.
> 
>         Also, those that want to shave off a few extra bytes in their
>         client, have the option of sending a request that only includes
>         the bits they want.  If you care about expiry leave it out, same
>         with flags, tags, cas id's, and anything else.  Plus you can
>         stream-line some of your requests by not using the CLEAR
>         command, and re-using the state.
> 
>         Dang, if I had a little more time on my hands right now, I'd be
>         really tempted to implement it.   I don't actually have a *need*
>         for this protocol in memcached, it was purely an intellectual
>         itch ever since I saw people complaining about the existing
>         protocols being difficult to expand.
> 
> 
> 
>         On Feb 20, 2008 4:14 PM, Dustin Sallings <dustin at spy.net
>         <mailto:dustin at spy.net>> wrote:
> 
> 
>             On Feb 19, 2008, at 22:20, Clint Webb wrote:
> 
>             > I know a considerable amount of thought and work has gone
>             into the
>             > existing flavour of the binary protocol, and I dont expect
>             that work
>             > to be discarded, I'm really only mentioning this new
>             concept now as
>             > an alternative for the future if we ever find the current
>             binary
>             > protocol to be too restrictive and inflexible.  And
>             something to
>             > think about, or even use elsewhere.
> 
>                    The is certainly interesting.  The first step of
>             doing the binary
>             protocol implementation was to create a change that allowed
>             multiple
>             protocols to coexist.  It would be possible to implement
>             this to run
>             in parallel with the existing protocols in the 1.3 codebase.
> 
>                    Intuitively, it doesn't seem as efficient to process
>             as what we've
>             got now, but I like being proven wrong, so I'd welcome
>             another front-
>             end protocol.  :)
> 
>                    Of course, I wrote the majority of the current binary
>             protocol code
>             about six months ago, so I'd really like to at least have
>             one in more
>             people's hands.
> 
>             --
>             Dustin Sallings
> 
> 
> 
> 
>         -- 
>         "Be excellent to each other" 
> 
> 
> 
> 
>     -- 
>     "Be excellent to each other" 
> 
> 
> ------------------------------------------------------------------------