PHP pecl/memcache with udp, consistent hashing, non-blocking-io and pipelining

[Mikael Johansson]

Hi,

I've had some questions about the future of pecl/memcache, so I thought
I might as well make some sort of announcement. The new NON_BLOCKING_IO
branch represents the next major milestone and contains some rather big
infrastructure changes; as such it should not be considered production
ready (although it runs the unit-tests without fail.)

Help with testing and performance tuning/benchmarking would be much
appreciated, especially in larger setups while using UDP. New features
include:

 * Non-blocking io where requests are queued onto servers and processed
in a select() loop; enables increased throughput since request
construction (serialize, compress, ..) and request parsing (deserialize,
decompress, ..) is now done in parallel with network send/recv. One
consequence is that performance should now increase as you add more
servers to the pool (before you might have seen a slight decrease.)
 * Request pipelining, requests are processed as they are built and as
responses are received, senders don't wait for readers and vice versa.
 * UDP support, currently only get requests use it per default but it
will be trivial to support UDP for other type of requests in the future.
 * Multi-key (and pipelined) set/delete/increment/decrement (about 10x
the performance of single-key requests when doing 50-keys per request)
 * Consistent hashing using crc32("host:port"), as opposed to using md5
or sha1, motivation being that crc32 is readily available and fast. Has
any standard been reached on this issue?

To maintain backwards-compatibility a new class has been introduced
(MemcachedPool) which encapsulates the new features. API is available at

 http://www.synd.info/extensions/memcache/api-changes-nonblocking.txt
 http://cvs.php.net/viewvc.cgi/pecl/memcache/?pathrev=NON_BLOCKING_IO

You can check out and build a copy as

 cvs -d :pserver:cvsread at cvs.php.net:/repository co -r NON_BLOCKING_IO
pecl/memcache
 cd pecl/memcache
 phpize
 ./configure
 make

Some early benchmarks comparing NON_BLOCKING_IO to HEAD

 http://www.synd.info/extensions/memcache/benchmark-20070522/

Benchmarks were done against 3 dual-cpu xeons each running 2 memcached
instances. The client was run on a 4th server, interconnected via
gigabit ethernet. Multi-key operations were done in batches of 50 to
simulate your average web-page.

Benchmark script:

http://cvs.php.net/viewvc.cgi/pecl/memcache/profile.php?pathrev=NON_BLOCKING_IO

//Mikael

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 250 bytes
Desc: OpenPGP digital signature
Url : http://lists.danga.com/pipermail/memcached/attachments/20070530/9a69ae23/signature-0001.pgp