binary protocol motivations
dustin at spy.net
Tue Nov 13 18:33:41 UTC 2007
On Nov 13, 2007, at 2:09 , Tomash Brechko wrote:
> On Mon, Nov 12, 2007 at 14:43:11 -0800, Marc wrote:
>> For the record, my motivation for a binary protocol was not
>> efficiency but more efficient I/O, especially for large sets of
>> small keys
>> with small values, AND to reduce code complexity.
> That's why you should be voting against "tags" approach. Now you have
> the reply format:
Yes, there's a 12-byte header on each response. It seems that you're
concerned about the overhead of the four bytes we use as a client
The concept of a client opaque to show a relationship between a
request and a response isn't exactly unprecedented (they are actually
called ``tags'' in IMAP or ``sequence numbers'' in TCP).
It is what allowed us to do partially-non-responsive commands (for
example, if we were to implement your multi-delete in the binary
protocol, we'd do it by means of a quiet delete which *may* still
error if it's a deferred delete).
At the very least, it certainly provides a tremendous comfort level
when I'm writing clients (assert resp->opaque == req->opaque kind of
On Nov 13, 2007, at 2:26 , Tomash Brechko wrote:
> So you'll save 4 bytes on positive response and a cost of 3 bytes on
> negative responce. With the hit rate > 43%, you'll have the saving.
That's only true if you do away with the unified response structure.
As it is, neither side has to do any processing at all until it sees
at least 12 bytes (at which point it knows what the message is, and
how large it's going to be).
For your alternative get-request-nak packet to work, we'd have to
know that this one response in this one context has a truncated
response wherein we won't get the length. I don't really see that
reducing code complexity, but even if it's worth the effort, you've
only ended up saving nine bytes in the case you're arguing shouldn't
be very common.
On Nov 13, 2007, at 10:19 , Tomash Brechko wrote:
> On Tue, Nov 13, 2007 at 10:08:14 -0800, Dustin Sallings wrote:
>> I see the point, I just don't see anything to justify the
>> criticality of your tone.
> See my mail where I say that if you have > 27.3% of hits on average,
> you better remove key id field, and send proper NAKs.
That's a bit disingenuous. Seriously, it's four bytes. You save a
meg every 256k hits and you did your math assuming it was OK to cut
off a large chunk of the NAK response. For people using gigE, they'd
have to have over 30 million hits before they have saved a cumulative
More information about the memcached