<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=US-ASCII">
<META NAME="Generator" CONTENT="MS Exchange Server version 6.5.7226.0">
<TITLE>Question concerning cpu utilization and memcached...</TITLE>
</HEAD>
<BODY>
<!-- Converted from text/rtf format -->
<P><FONT SIZE=2 FACE="Arial">Environment: FreebBSD 4.9, memcached-1.1.12, libevent 1.0c, python-memcached-1.2</FONT>
<BR><FONT SIZE=2 FACE="Arial">Issue: Very heavy CPU utilization during get() processing</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">I googled around and didn't find anything useful on this, so please be gentle if this question is a repeat of a previous discussion. :-)</FONT></P>
<P><FONT SIZE=2 FACE="Arial">I've spent the last couple days building out a memcached-based front end for one of our systems. It caches a little over 1 million items, where the key is fairly short, and the content is also very short. We query a good sized database and load everything in cache, and then run all lookups out of cache. Performance has been excellent. In testing in a grid configuration with 30+ nodes I am getting 500-600 cache hits per second on all nodes simultanesously which is really good (especially considering that we're using Python to drive the process). I do observe one troubling behavior that I'm curious to know more about. </FONT></P>
<P><FONT SIZE=2 FACE="Arial">I configured memcached to run on 9 servers at 250 mb/server. When the cache is loaded, it looks like all instances are fairly evenly loaded (memory stats are virtually identical for all instances). However, when we start running get()s I note that 1 of the 9 machines gets extremely busy in terms of CPU utilization, while the rest of the machines load fairly evenly. For example, in one stress test I ran, the CPU utilization across 8 nodes averaged 5-6%. One node, however, had utilization approaching 80%. I am worried about having CPU utilization issues deploying this into production as we are running many CPU intensive processes on the same set of machines. I also noticed this same behavior early on testing with just two instances of memcache running.</FONT></P>
<P><FONT SIZE=2 FACE="Arial">Has this type of behavior been observed by others? Any information as to what causes this or how to avoid it?</FONT>
</P>
<P><FONT SIZE=2 FACE="Arial">Thanks for any insight….</FONT>
<BR><FONT SIZE=2 FACE="Arial">Chris Gillett</FONT>
</P>
</BODY>
</HTML>