libevent and epoll

[Brad Fitzpatrick]

Perrin,

I've been having fun playing with this, and I'll continue to do so later,
but I gotta run now.  A couple notes first, though:

  * Your connect to MySQL is over "localhost", which the mysql client
    remaps to using a Unix socket.  Change it to using TCP and watch DBD
    perf drop quite a bit.  (In real life, web/db/cache nodes are all
    separate for best CPU/mem/network usage)

  * You're ignoring concurrency issues.  Increase your clients to more
    than 1 and watch DBD performance drop because MySQL starts locking.
    Sure, you could use InnoDB (concurrent, but slow) or HEAP tables,
    but HEAP tables are very limiting.

  * Your dataset is so small, MySQL and the kernel buffer cache never
    actually writes.  That'll hurt you more than anything.  I haven't
    tested this yet, but I've seen it enough in real life (before
    memcache) that I know it's painful.  (your test case is also
    sequential, which means MySQL likely isn't doing random reads,
    even if it weren't already all in memory...)

That said, the DBD client library is faster at network I/O and parsing
than MemCachedClient.pm, but that's understandable since it's written in
C, and MemCached's is pure Perl.  We could always make a C version, but I
don't think it'd be worth it.  I'll work on tuning the Perl to be faster
first.

I'll keep playing with this benchmark and help you improve it if you like,
but that'd involve making it much more real-life-like.  As it is, it isn't
measuring anything too applicable to real high-traffic usage.

- Brad


On Sun, 28 Sep 2003, Perrin Harkins wrote:

> On Sun, 2003-09-28 at 14:51, Brad Fitzpatrick wrote:
> > Whoa, that's incredibly slow.  I'm guessing you're using poll with a buggy
> > libevent.
>
> I'm using the statically compiled memcached you posted, so hopefully any
> stray libevent libraries on my system would not affect it.  Running
> locate doesn't show anything called libevent installed on my system.
>
> > What method is libevent using?
>
> It says "using epoll" when I start it.  Is it possible that the 2.4.21
> kernel has issues with epoll?  The patch applied cleanly.
>
> > Try this binary with those fixes:
> >
> > http://www.danga.com/memcached/dist/binaries/linux-x86/memcached-1.1.9-snapshot3.gz
>
> I get the same speed with this one.
>
> > In the end, though, I don't much care if memcached is a few percent slower
> > than shared memory on a single machine.
>
> I totally agree, but it needs to be faster than MySQL and in my current
> test it isn't.
>
> > Can you post your entire test harness so I can run it myself?
>
> Sure, I'll attach it, with the MySQL code for comparison.
>
> - Perrin
>