Fast writes, slow reads

[Brion Vibber]

Brad Fitzpatrick wrote:

>It's even slow when you're using kqueue?!  That's supposed to be
>[slightly] faster than epoll.
>
>Was it slow with kqueue and the Perl client or the PHP client?
>

I have tried combinations of the following:

ports-patched 1.1.6 server on FreeBSD, using kqueue
self-compiled 1.1.8 server on Linux, using poll
dl'd binary 1.1.8 server on Linux, using poll
(I haven't yet managed to get a kernel recompiled with the epoll patch.)

with

PHP client w/ PHP 4.3.2 on BSD
Perl client w/ Perl 5.6.1 on BSD
Perl client w/ Perl 5.8.0 on Linux
Python client w/ Python 2.2.1 on Linux
Python client w/ Python 2.2.2 on BSD

So far I've consistently found that:

* gets are reasonably fast when there's no data to return

* sets, adds, replaces, and deletes are reasonably fast, whether or not 
the key already exists

* gets have a large fixed overhead when there is data to return, which 
is consistent across all three client libraries and server connections 
but varies according to the client's system (0.10 seconds on 2GHz 
Athlon/FreeBSD, 0.04 seconds on 266MHz PII/Linux)


With Evan's "slow.py" demo, I see the wait disappear after the first get 
when using a delay longer than the minimum time on the Linux box, 
connecting to either server:

$ python slow.py 0.03
Trying get()s with 0.03s delay
get took 0.04 seconds
get took 0.04 seconds
get took 0.04 seconds
...

$ python slow.py 0.05
Trying get()s with 0.05s delay
get took 0.04 seconds
get took 0.00 seconds
get took 0.00 seconds
...

But it never goes away on the BSD box, connecting to either server:
$ python slow.py 1
Trying get()s with 1.00s delay
get took 0.10 seconds
get took 0.10 seconds
get took 0.10 seconds
...

>One problem we run into every couple months at LiveJournal is new
>network hardware coming up in the wrong duplex.  Auto-negotiation never
>seems to work that well.  Can you verify both sides are in the same
>duplex, either half or full?
>

I get the same performance (within my testing tolerance, anyway) over 
loopback as I do when talking to the other box over the network...

-- brion vibber (brion @ pobox.com)