PHP pecl/memcache with udp, consistent hashing, non-blocking-io
mikael at synd.info
Fri Jun 1 18:39:23 UTC 2007
Support for libevent would be nice but since the client wouldn't have
more than N sockets active at any one time, while the server have a
potential N*M sockets open (where N is the number of servers and M is
the number of potential clients (eg. webservers * max-processes)) and it
requires extra libraries it is not prio 1.
I'm also not currently aware as to how this might affect the current
architecture, as I understand it libevent uses callbacks to provide
event notification? It might require much #ifdef's to support both
select() and libevent schemes.
Memcached sends udp responses in 1400 byte chunks including a header and
pecl/memcache glues them together before returning to the caller;
however with large multi-packet responses the possibility of dropped or
out-of-sequence packets increases which makes pecl/memcache back off udp
for a few seconds and revert to tcp. Have you had problemes with
responses larger than 20kb, ie. "E_NOTICE: udp packet loss, expected .."
> How about using libevent for non-blocking io instead of select()?
> libevent is more effective than select().
> From my test maxium size of single udp datagram can transfer
> successfully is about 20K(not 64k) for local network.
> Item which value size is bigger than 20k will split into several
> datagrams, pecl/memcache extension has to
> re-assemble them. Situation becomes even worse when some of split
> datagrams lost during transmittion.
> Que Hongyu
> On 5/31/07, *Mikael Johansson* <mikael at synd.info
> <mailto:mikael at synd.info>> wrote:
> I've had some questions about the future of pecl/memcache, so I thought
> I might as well make some sort of announcement. The new NON_BLOCKING_IO
> branch represents the next major milestone and contains some rather big
> infrastructure changes; as such it should not be considered production
> ready (although it runs the unit-tests without fail.)
> Help with testing and performance tuning/benchmarking would be much
> appreciated, especially in larger setups while using UDP. New features
> * Non-blocking io where requests are queued onto servers and processed
> in a select() loop; enables increased throughput since request
> construction (serialize, compress, ..) and request parsing
> decompress, ..) is now done in parallel with network send/recv. One
> consequence is that performance should now increase as you add more
> servers to the pool (before you might have seen a slight decrease.)
> * Request pipelining, requests are processed as they are built and as
> responses are received, senders don't wait for readers and vice versa.
> * UDP support, currently only get requests use it per default but it
> will be trivial to support UDP for other type of requests in the
> * Multi-key (and pipelined) set/delete/increment/decrement (about 10x
> the performance of single-key requests when doing 50-keys per request)
> * Consistent hashing using crc32("host:port"), as opposed to using md5
> or sha1, motivation being that crc32 is readily available and fast. Has
> any standard been reached on this issue?
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 250 bytes
Desc: OpenPGP digital signature
Url : http://lists.danga.com/pipermail/memcached/attachments/20070601/de9e1246/signature.pgp
More information about the memcached