binary protocol notes from the facebook hackathon

[Roy Lyseng]

Steve Grimm wrote:
> On 7/11/07 12:45 AM, "marc at corky.net" <marc at corky.net> wrote:
>> While network byte ordering (Big Endian) is traditionally the 'right'
>> thing to do (or the default thing to do), in most cases it's a minor
>> performance hit due to constant swapping.  Since we're implementing a
>> binary protocol specifically to avoid/minimize minor performance hits
>> and since this is a brand new protocol I would recommend to keep all
>> values as Little Endian because:
> 
> Defining a protocol that uses something other than network byte order gives
> me the willies. But I'm not sure I can argue with your reasoning on anything
> more than aesthetic grounds; swapping bytes around unnecessarily is exactly
> the sort of thing we're trying to avoid.
> 
> I wonder just what the overhead of byte swapping will be, though, compared
> to all the other work that has to happen to handle a request. It may be a
> small enough expense to disappear into the statistical noise. If we'd be
> saving .001% of the CPU time by avoiding swapping, I'd say forget it, not
> worth the confusion.  I've never actually measured the cost of using network
> byte order in any of my network apps, though, so I honestly have no idea.

I have not actually measured this, but I do not think this is a problem 
on modern computers. Getting the bytes into the cache line is probably 
the biggest time consumer. And unless you rely on an aligned byte 
offset, you will have to do some byte fiddling anyway.
> 
> If it's a large overhead, maybe we can make endianness depend on the magic
> number at the start of the command. It is annoying to support both, granted,
> but that would potentially allow a client to be configured to send requests
> and receive responses in whatever format is most efficient for a particular
> server. (Or vice versa, I suppose.) Or would the overhead of having to
> support both styles eat all the cycles you'd save by not swapping bytes? I'm
> not sure.
> 
> I suspect that the Sun folks would appreciate not having to flip bits back
> to network byte order on their SPARC boxes, but I agree with you that Intel
> and AMD totally dominate memcached's target audience at the moment.
> Supporting both orders from the start would insulate us from changes in that
> market reality, though maybe that's far enough off that it's better to just
> rev the protocol if/when the need arises.

We deliver AMD and Intel boxes these days also :)

However, I would vote for preserving network byte order, because I do 
not think there is a real gain in switching this strategy, not 
mentioning having two different strategies in one protocol.

Roy

> 
> -Steve
>