benchmark for single-threaded vs. multi-threaded memcached

[Steven Grimm]

I don't think that's a fair test since the server-side application code
is not the same. spcached might be significantly less optimized than
memcached which will reduce your throughput for reasons that have
nothing to do with multithreading.

Please try memcached 1.2.2 compiled in both single-threaded and
multi-threaded modes; the code paths there are not 100% identical but
they're a lot closer to identical than in this test. To compile 1.2.2 in
MT mode, run "./configure --enable-threads" when you build it. You can
control the number of threads with the "-t" option. Note that each
client connection is bound to a thread, so make sure your benchmark
opens lots of connections if you want to do a fair test of thread
performance.

In addition, you might want to compare the single-threaded memcached
1.2.2 with the same version compiled as a multi-threaded app, but run
with "-t 1" so that it only uses one thread. That will tell you how much
overhead is introduced by the threading infrastructure itself (acquiring
locks, etc.) without any possibility of resource contention affecting
the results.

-Steve


liusifan wrote:
> Hi, all
> I've make a benchmark for single-threaded vs. multi-threaded
> memcached. The single-threaded program is memcached 1.2.0.
> The multi-threaded program is spcached.
> spcached is a server which implements memcached protocol.
> More detail about spcached, please refer its home page:
> http://code.google.com/p/spcached/
> Using memcached java client as benchmark tool:
> http://code.google.com/p/spcached/wiki/benchmarktool
> bash$ java com/danga/MemCached/test/MemCachedThreadBench
> <runs> <start> <port> <threads>
> memcached 1.2.0 is a single-threaded event-driven program.
> spcacheduses a half-sync/half-async thread pool, it has a main
> thread to process I/O event, and a worker thread to process
> the real request.
> The test enviroment is :
> Hardware: DELL 640M ( CPU Intel T2300, RAM 1G )
> Software: windows xp
> vmware 5.5.1 ( allocate 128M RAM )
> RedHat 7.2(Kernel 2.4.18-3)
> memcached 1.2.0
> spcached svn revision 5 <http://code.google.com/p/spcached/source> +
> spserver 0.3
> <http://freshmeat.net/projects/spserver/?branch_id=68862&release_id=253604>
> + spdict 0.2
> <http://freshmeat.net/projects/spdict/?branch_id=69457&release_id=253599>
> The test steps are:
> 1.restart windows xp and vmware
> 2.limit memcached and spcached both can only keep 185808 objects
> 3.start memcached and MemCachedThreadBench
> ( the memcached and benchmark tool run on the same vmware )
> bash$ memcached -m 96
> 4.stop memcached, start spcached and MemCachedThreadBench
> bash$ spcached -c 185808
> The test result is:
> For set operation:
> memcached : spcached -- 4000 : 2600 ( > 4 threads )
> For get operation:
> memcached : spcached -- 3500 : 2480 ( > 4 threads )
> Detail test result:
> memcached
> PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
> 22642 xxxxx 15 0 100M 60M 5488 S 0.0 49.2 19:27 memcached
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 500000 0 11211 1
> Thread start runs set time(ms) get time(ms)
> Main 500000 168924 149234
> ReqPerSecond set - 2959 get - 3350
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 250000 0 11211 2
> Thread start runs set time(ms) get time(ms)
> Main 500000 135051 153600
> ReqPerSecond set - 3702 get - 3255
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 166666 0 11211 3
> Thread start runs set time(ms) get time(ms)
> Main 499998 141120 156591
> ReqPerSecond set - 3543 get - 3193
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 125000 0 11211 4
> Thread start runs set time(ms) get time(ms)
> Main 500000 123370 142169
> ReqPerSecond set - 4052 get - 3516
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 100000 0 11211 5
> Thread start runs set time(ms) get time(ms)
> Main 500000 124111 142147
> ReqPerSecond set - 4028 get - 3517
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 83333 0 11211 6
> Thread start runs set time(ms) get time(ms)
> Main 499998 123504 141857
> ReqPerSecond set - 4048 get - 3524
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 71428 0 11211 7
> Thread start runs set time(ms) get time(ms)
> Main 499996 124767 142939
> ReqPerSecond set - 4007 get - 3497
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 62500 0 11211 8
> Thread start runs set time(ms) get time(ms)
> Main 500000 124746 142887
> ReqPerSecond set - 4008 get - 3499
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 55555 0 11211 9
> Thread start runs set time(ms) get time(ms)
> Main 499995 124821 143278
> ReqPerSecond set - 4005 get - 3489
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 50000 0 11211 10
> Thread start runs set time(ms) get time(ms)
> Main 500000 124865 143082
> ReqPerSecond set - 4004 get - 3494
>
> =============================================================================
> spcached
> PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
> 24536 xxx 15 0 74948 73M 608 S 0.0 59.2 25:31 spcached
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 500000 0 11216 1
> Thread start runs set time(ms) get time(ms)
> Main 500000 195136 209098
> ReqPerSecond set - 2562 get - 2391
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 250000 0 11216 2
> Thread start runs set time(ms) get time(ms)
> Main 500000 199820 213581
> ReqPerSecond set - 2502 get - 2341
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 166666 0 11216 3
> Thread start runs set time(ms) get time(ms)
> Main 499998 190134 203171
> ReqPerSecond set - 2629 get - 2460
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 125000 0 11216 4
> Thread start runs set time(ms) get time(ms)
> Main 500000 189092 202166
> ReqPerSecond set - 2644 get - 2473
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 100000 0 11216 5
> Thread start runs set time(ms) get time(ms)
> Main 500000 189452 200893
> ReqPerSecond set - 2639 get - 2488
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 83333 0 11216 6
> Thread start runs set time(ms) get time(ms)
> Main 499998 188704 201101
> ReqPerSecond set - 2649 get - 2486
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 71428 0 11216 7
> Thread start runs set time(ms) get time(ms)
> Main 499996 189220 201354
> ReqPerSecond set - 2642 get - 2483
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 62500 0 11216 8
> Thread start runs set time(ms) get time(ms)
> Main 500000 190293 201664
> ReqPerSecond set - 2627 get - 2479
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 55555 0 11216 9
> Thread start runs set time(ms) get time(ms)
> Main 499995 191152 202004
> ReqPerSecond set - 2615 get - 2475
> --
> bash$ java com/danga/MemCached/test/MemCachedThreadBench 50000 0 11216 10
> Thread start runs set time(ms) get time(ms)
> Main 500000 191352 202574
> ReqPerSecond set - 2612 get - 2468
> ------------------------------------------------------------------------
> liusifan
> 2007-05-18
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/memcached/attachments/20070517/4c768f83/attachment.html