libevent and epoll

[Brad Fitzpatrick]

Whoa, that's incredibly slow.  I'm guessing you're using poll with a buggy
libevent.

What method is libevent using?  Niels Provos has been fixing some bugs in
libevent lately, regarding both epoll and poll.

Try this binary with those fixes:

http://www.danga.com/memcached/dist/binaries/linux-x86/memcached-1.1.9-snapshot3.gz

You should benchmark it using epoll, which is incredibly fast, but poll
should even be way faster than what you measured.

In the end, though, I don't much care if memcached is a few percent slower
than shared memory on a single machine.  The whole point of memcached is
that it's distributed.  No other solution would allow LJ to have 40 GB of
redundant, shared cache.

Can you post your entire test harness so I can run it myself?

- Brad



On Sun, 28 Sep 2003, Perrin Harkins wrote:

> On Tue, 2003-09-23 at 00:28, Brad Fitzpatrick wrote:
> > And use our binary, which is built with a libevent with epoll support,
> > including epoll bugfixes which aren't in 0.7a (the latest libevent version
> > on the site).
> >
> > http://www.danga.com/memcached/dist/binaries/linux-x86/memcached-1.1.9-snapshot-dbg.gz
> >
> > Test it with:
> >
> > $ EVENT_SHOW_METHOD=1 ./memcached
> >
> > It should say "Using: epoll" or similar.
>
> Thanks for the help.  I got this working (on linux kernel 2.4.21), but
> I'm getting pretty poor performance so I think I must have a mistake in
> my code somewhere.  At the moment, MySQL, IPC::MM, and BerkeleyDB are
> all much faster than memcached in my tests.
>
> I'm starting the server with this command:
>
> ./memcached-1.1.9-snapshot-dbg -d -m 128 -l 127.0.0.1 -p 11211
>
> Here's the client code I use.  IPC::SharedHash is an asbtraction layer
> that allows me to test all forms of storage with the same API.
>
> package IPC::SharedHash::MemCachedClient;
>
> use strict;
> use warnings;
>
> use MemCachedClient;
>
> sub new {
>     my $class   = shift;
>     my $self    = {};
>     my $memc    = MemCachedClient->new(
>         {
>             'servers'            => ['127.0.0.1:11211'],
>             'debug'              => 0,
>             'compress_threshold' => 10_000,
>         }
>     );
>
>     $self->{'backend'} = $memc;
>     bless $self, $class;
>     return $self;
> }
>
> sub fetch {
>     my $self  = shift;
>     my ($key) = @_;
>     my $value = $self->{'backend'}->get($key);
>     return $value;
> }
>
> sub store {
>     my $self = shift;
>     my ( $key, $value ) = @_;
>     $self->{'backend'}->set( $key, $value );
> }
>
> 1;
>
> Does anyone see any problems here?  By the way, lines 305 and 306 of the
> Perl client API were giving warnings, but neither turning off warnings
> nor fixing the problem made a significant difference to performance.
> Here's the output of my benchmark script:
>
>                   Rate MemCachedClient      DBD::mysql         IPC::MM
> MemCachedClient 16.4/s              --            -53%            -85%
> DBD::mysql      35.0/s            114%              --            -68%
> IPC::MM          109/s            564%            210%              --
>
> - Perrin
>
>