binary protocol notes from the facebook hackathon

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

marc at corky.net wrote:
> Hi everyone,
> 
> I'm happy to see a nice and compact result with zero bloat.  I'm also 
> happy you guys kept alignment within the request/response struct and 
> that would help performance.
> 
> I see byte ordering is mentioned twice;  the length field both in the 
> request and response.
> 
> While network byte ordering (Big Endian) is traditionally the 'right' 
> thing to do (or the default thing to do), in most cases it's a minor 
> performance hit due to constant swapping.  Since we're implementing a 
> binary protocol specifically to avoid/minimize minor performance hits 
> and since this is a brand new protocol I would recommend to keep all 
> values as Little Endian because:
> 
> - It's easier that all values are kept to a the same endianess; reduces 
> confusion.

Heh, agreed.  Any numeric value larger than one byte should be in
network order, which removes the confusion. ;)

> - Nowadays MOST (but obviously not all) servers are running little 
> endian.   So this saves byte swapping for most people's cases and thus a 
> few cycles are spared on each request -- isn't that the whole point? ;)

- -1.  Burden of proof is on those wanting host order to show *measured*
overhead on real workloads.

> And since I mentioned alignment at the top;  Would the entire packet, 
> including its payload, be aligned?   It can be a waste of up to a three 
> bytes per stored object but could potentially improve performance a 
> little bit -- something entertaining to benchmark -- think we'll be able 
> to notice a timing difference above the noise level? :)

- -1 for aligning the payload on the wire;  you can't reasonably expect to
get the whole request in a single packet, anyway.  The server is of
course free to pad for storage, although I doubt it will help anything.
Averaging 2 extra bytes per key won't matter for "rocky" slabs (i.e.,
fillee with "big" values), but would be *terrible* for "gravelly" ones.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGlPjY+gerLs4ltQ4RAtHbAJwKa1wWXt4/9F4ypnHjivFmLBQ7awCfSkM9
/TLJzkVkYLrYe/NfMvtEpvo=
=7nsY
-----END PGP SIGNATURE-----