Occasional slowness, hitting select_timeout in Cache::Memcached

[Alex Greg]

Hi,

We're using memcached and are generally very happy with it - it's
serving over 2,500 requests per second on one server (Fedora Core 3,
2.6 kernel w/epoll), and we've generally found it to be very fast and
a great way to offload queries from our database server.

There is only one nagging problem; very occasionally (~0.1% of
queries), we see the memcache server taking too long to send the data,
which means the client wait for 1 second (the value of select_timeout
in Cache::Memcached) before declaring a cache miss and getting the
data from the database. I've reduced select_timeout to 0.2 secs to
reduce the impact of this problem, but it still persists (albeit
taking 0.2 seconds before declaring a cache miss).

What could be causing this occasional slowness? It doesn't seem to be
on any particular key, and seems to occur completely at random. The
client and server machines are all running Linux 2.6 with epoll.

Regards,

-- Alex