memcached-1.1.12 does infinite loop

[Anatoly Vorobey]

On Sun, May 29, 2005 at 04:38:57PM +0200, Andreas Vierengel wrote:
> I can confirm this behaviour. I have recently tested get_multi with a
> variable amount of keys. On my 2.8 GHz Xeon machine it took 0.8s so
> process a get_multi with 10.000 keys. During this time, nothing else is
> done by memcached. ( I have also done 500.000 keys and this stalls
> everything for minutes ).
> About 1% of my requests do get_multi with about 1000 keys, so I'm going
> to limit one get_multi to about 500 keys, too.
> 
> After looking at the code, I think the granularity of an event is one
> command. If the command is a 500.000-keys-get_multi memcached is forced
> to satisfy this request before continuing with other work.

The intention of the code has always been: if you get new input on a
connection, process it and write out the output for its request for 
as long as you can do it without waiting, i.e. until you either block at 
reading (input is over) or block at writing (the TCP stack is slower 
than you, even on a LAN).

When memcached gets a very large get_multi, here's what *should* happen:
it very quickly finds all the items that are requested and puts
pointers to them into a large buffer array; then it asynchronously goes
over the array and sends out the responses with the data, yielding
to other requests on other connections when it can't write immediately.

If there's a large delay on a very large get_multi, it could be due
to: either the "very quickly" part above is not as optimised and fast 
as it should be (I certainly never imagined it could take minutes
on 500,000 keys), or perhaps the writing part is bugged and blocks at
some place it shouldn't block. I will try to investigate.

> Another idea is to rewrite portions of the memcached-code, so that
> event-handling is not only done for async network I/O, but also within a
> get_multi-request. Maybe in the event of a writeable socket, the
> get_multi fetches more key/values from this request until the socket
> write-buffer is full. The "state-machine" must be enhanced for this, I
> think. Buffering the whole response, like it is done at the moment, will
> also not be needed.

The whole response is not buffered; only an intermediate list of 
pointers to items that have been located (the creation of this list 
doesn't involve network I/O) is buffered. The state-machine *is* used 
for asynchronously building and sending out the actual response with the 
data.

-- 
avva
"There's nothing simply good, nor ill alone" -- John Donne