memcached - first impressions and issues

[Anatoly Vorobey]

On Thu, Dec 18, 2003 at 06:29:21PM -0600, Andy Bakun wrote:
> The PHP API, linked from the memcached web page, works great, although
> the lack of more complete sample code caused a few misstarts on my
> part.  It appears to have a problem with not returning right away if it
> can't connect, so it's difficult to gracefully recover, but I need to
> look into this more (perhaps I'm missing an option?).
> 
> One problem I've noticed is that the protocol docs mention a flush_all
> command, which doesn't seem to exist in the 1.1.9 distribution.

It exists in the CVS tree, but it was added after 1.1.9 had been 
released, so it'll go into the next release. If you need it, you can 
take the latest CVS version, it's extremely stable. I suppose we should 
release a new version soon; a good time for this would be 
after I finish an overhaul of the Perl client (I'm rewriting it to use
asynchronous IO using select() on many sockets simultaneously, when 
using >1 memcache server, rather than slower and, as it turned out, less 
reliable synchronous IO with alarm() as a timeout mechanism). 

> Also, are there any plans for additional reporting features?  One thing
> I'm in need of is a way to list all the keys that are currently cached,
> and their expiration time.  Along with this, I'd like to be able to
> delete keys that match a certain pattern (although I can certainly do
> this myself if I can get a list of all keys).

Both these features are possible, but seem to go against the design of 
memcached in several ways. If you want to get a complete list of all 
keys synchronously, a momentary snapshot, the server would have to 
suspend dealing with all other clients for a rather long time (seconds?)
to gather and transmit all this data - a lot of data for a medium to 
large memcached server. If you don't mind other clients working at the 
same time, your snapshot will not be momentary.

There's an existing debug command, "stats cachedump" (perhaps not 
documented in the protocol, I don't remember - it's not meant to be 
stable or official) which allows you to list first N keys in a 
particular LRU queue (there're about 15 LRU queues, each for a different 
size class), where N could also be "all of them". This gives you a 
momentary snapshot. However, it's not a stable command, and 
its meaning/syntax will change at some point when we redesign the 
LRU structure of memcached. 

Deletion of keys that match a certain pattern will probably be 
CPU-intensive because the server would have to loop over all the keys, 
which it currently never does (it finds keys using hash tables, which 
would be useless here). memcached is generally very very light on CPU 
usage, so we might be reluctant to introduce such a feature.

-- 
avva