libevent and epoll

[Brad Fitzpatrick]

> >   * Your connect to MySQL is over "localhost", which the mysql client
> >     remaps to using a Unix socket.  Change it to using TCP and watch DBD
> >     perf drop quite a bit.
>
> I will try that.  I thought it only used unix sockets if I told it to at
> compile time.

Unix sockets are the default.  In Debian, at least, you have to explicitly
turn networking on.  If Unix sockets are available, TCP won't even be
used.  Both localhost and 127.0.0.1 are assumed to mean "fastest way to
connect to local machine".

> thing up on CPAN for people to play with after that.  (By the way, how
> about putting MemCachedClient on CPAN?  Maybe as IPC::Memcached or
> something?)

Don't know how.  *shrug*  Guess it'd be easy to find.

> Having said that, I found that concurrency didn't make very much
> difference.

Having run a site with over a dozen MySQL servers, I found the opposite.  :)

> Even in the case of things like MLDBM::Sync, which lock the
> whole database during writes, relative performance of these tools was
> about the same with many concurrent clients as it was in the single
> client test.

Keep going.  More clients (from more machines).  You will hit a wall.

> In my experiments with MySQL, heap tables have about 50% better
> performance, but they can't store anything larger than a varchar 256 so
> they're basically useless.

And the 4 GB virtual memory limit.

> >   * Your dataset is so small, MySQL and the kernel buffer cache never
> >     actually writes.
>
> Yes, this is true.  I use a larger data set in the other test.  Even so,
> my goal is not to use more data than can fit in memory.  You wouldn't
> recommend that people run memcached in a way that would cause it to
> swap, would you?

Memcached won't swap, if you use the -k option.  It throws out old keys
that haven't been accessed in awhile.  (another thing MySQL won't do)

What I'm saying is that MySQL (well, MyISAM) is inherently disk-based, and
you can't tell it not to use the disk.  You can control the index cache,
but you can't tell it throw stuff out and never go to disk.  And you have
no control over the buffer cache, unless you use InnoDB.  And once you go
to disk, all hell breaks loose and you start blocking and locking.

> > I'll work on tuning the Perl to be faster first.
>
> I plan to run it through the profiler later on and see if there's
> anything obvious to tune.

dprofpp won't be too useful, since there aren't many subs in the client
library.  I think I just need to change the result parsing and do I/O in
bigger chunks.

I'm home now... I'll see what I can do.  I'll either work on speeding up
the client, or make a big feature/performance grid of all the caching
options with notes.

- Brad