automatic upgrade of individual gets to multi-key gets

[Dustin Sallings]

	In one of my applications, an HTTP request typically translates to a  
single memcached lookup.  A page load can have twenty or so images  
that are dynamically(-ish) generated and may be served out of  
memcached, but they are individual requests.

	I was thinking about how I might express this in terms of a multi- 
keyed get and the obvious place to do this seemed to be in my  
memcached client itself <http://bleu.west.spy.net/~dustin/projects/ 
memcached/>.

	I added an optimization such that when I'm pulling a request from  
the read queue, if it's a get request, I check to see if there's  
another get request after it.  If so, I collapse all of the  
sequential get requests to a server into a single request for all of  
the distinct keys from the client and properly distribute the results  
back to the callers.

	In the case where a given key is accessed concurrently frequently,  
the value only needs to be transferred once per concurrent access.

	Note that this will never introduce a delay in processing, since it  
only occurs while reading from the request queue.


	I deployed this in my app and processed 44 distinct gets in 114ms  
right away.  This is approximately 10x faster than my original  
implementation.  I find that my cache access performance increases  
over time (jit or whatever) so I expect it to approach negligible  
soon (my min wait time is already 1ms).

-- 
Dustin Sallings