binary protocol motivations

Dustin Sallings dustin at
Tue Nov 13 18:33:41 UTC 2007

On Nov 13, 2007, at 2:09 , Tomash Brechko wrote:
> On Mon, Nov 12, 2007 at 14:43:11 -0800, Marc wrote:
>> For the record, my motivation for a binary protocol was not  
>> computational
>> efficiency but more efficient I/O, especially for large sets of  
>> small keys
>> with small values, AND to reduce code complexity.
> That's why you should be voting against "tags" approach.  Now you have
> the reply format:

	Yes, there's a 12-byte header on each response.  It seems that you're  
concerned about the overhead of the four bytes we use as a client  

	The concept of a client opaque to show a relationship between a  
request and a response isn't exactly unprecedented (they are actually  
called ``tags'' in IMAP or ``sequence numbers'' in TCP).

	It is what allowed us to do partially-non-responsive commands (for  
example, if we were to implement your multi-delete in the binary  
protocol, we'd do it by means of a quiet delete which *may* still  
error if it's a deferred delete).

	At the very least, it certainly provides a tremendous comfort level  
when I'm writing clients (assert resp->opaque == req->opaque kind of  

On Nov 13, 2007, at 2:26 , Tomash Brechko wrote:

> So you'll save 4 bytes on positive response and a cost of 3 bytes on
> negative responce.  With the hit rate > 43%, you'll have the saving.

	That's only true if you do away with the unified response structure.   
As it is, neither side has to do any processing at all until it sees  
at least 12 bytes (at which point it knows what the message is, and  
how large it's going to be).

	For your alternative get-request-nak packet to work, we'd have to  
know that this one response in this one context has a truncated  
response wherein we won't get the length.  I don't really see that  
reducing code complexity, but even if it's worth the effort, you've  
only ended up saving nine bytes in the case you're arguing shouldn't  
be very common.

On Nov 13, 2007, at 10:19 , Tomash Brechko wrote:

> On Tue, Nov 13, 2007 at 10:08:14 -0800, Dustin Sallings wrote:
>> 	I see the point, I just don't see anything to justify the
>> 	criticality  of your tone.
> See my mail where I say that if you have > 27.3% of hits on average,
> you better remove key id field, and send proper NAKs.

	That's a bit disingenuous.  Seriously, it's four bytes.  You save a  
meg every 256k hits and you did your math assuming it was OK to cut  
off a large chunk of the NAK response.  For people using gigE, they'd  
have to have over 30 million hits before they have saved a cumulative  

Dustin Sallings

More information about the memcached mailing list