binary protocol notes from the facebook hackathon
Marc at facebook.com
Wed Jul 11 17:33:55 UTC 2007
I think little-endian wire protocol violates the principle of least
astonishment. One of the things we considered when going through this is
³will this be impossible to read with a protocol analyzer², I¹d rather see
00 00 00 01 than remember that 01 00 00 00 is a 4 byte, little endian field.
Also, the cost of byte swapping is absolute noise compared to just doing a
memory fetch and store. Whether I do
foo->length = htonl(*(long*)(buf+4));
foo->length = *(long*)buf+4
The cost is in the * and the =, not the htonl.
I don¹t think alignment is all that important. Values are byte aligned
anyway and for the header the ability to cast directly from header-buffer to
a struct is unsafe, unportable, and of unmeasurable performance gain.
We discussed word alignment briefly, but since values may not be aligned
we¹d have to add an additional field to indicate the number of pad bytes and
that would have to be word-al
On 7/11/07 12:45 AM, "marc at corky.net" <marc at corky.net> wrote:
> Hi everyone,
> I'm happy to see a nice and compact result with zero bloat. I'm also
> happy you guys kept alignment within the request/response struct and
> that would help performance.
> I see byte ordering is mentioned twice; the length field both in the
> request and response.
> While network byte ordering (Big Endian) is traditionally the 'right'
> thing to do (or the default thing to do), in most cases it's a minor
> performance hit due to constant swapping. Since we're implementing a
> binary protocol specifically to avoid/minimize minor performance hits
> and since this is a brand new protocol I would recommend to keep all
> values as Little Endian because:
> - It's easier that all values are kept to a the same endianess; reduces
> - Nowadays MOST (but obviously not all) servers are running little
> endian. So this saves byte swapping for most people's cases and thus a
> few cycles are spared on each request -- isn't that the whole point? ;)
> And since I mentioned alignment at the top; Would the entire packet,
> including its payload, be aligned? It can be a waste of up to a three
> bytes per stored object but could potentially improve performance a
> little bit -- something entertaining to benchmark -- think we'll be able
> to notice a timing difference above the noise level? :)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the memcached