Question re: General Performance Stats

[Brian Moon]

Andrew B. Westmoreland wrote:
> Hello -
> 
> We are well into testing a memcached as a means to offload read requests 
> from our database.  We are logging each query result to disk, and have 
> seen an interesting trend:
> 
> It seems that for a lot of the very small db queries, there is an 
> increase in latency when we pull the data from the memcache.  Whereas a 
> simple "select a from b where primary key ='x' " was taking ~300ms to 
> get from our database over a simple (and very busy) GigE switch, the 
> memcache is running on localhost and is seeing "query" times more like 
> ~600ms.
> 
> We tested it on many batch runs, and it seems that using the memcache 
> (as configured) causes a performance hit of approximately 20%.  This is 
> for a CPU-bound process that uses 100% of all CPU cores in both testing 
> scenarios.
> 
> I wanted to check with the group and see if this was typical when making 
> the transition.  I didn't see any mention of performance decrease in the 
> literature, so wanted to reach out to those of you who are using it and 
> working on it every day.  There is obviously real risk that the delay is 
> due to the way either the daemon is implemented on our systems, OR in 
> the way we implemented the cursor emulation in our code.

MySQL (assuming that is your db) is very, very good at primary key look 
ups.  If all your queries are primary key lookups, I am not sure 
memcached can help you.

However, for us, we received the most improvement by wrapping several 
queries into one piece of cache.  For us, and I would assume anyone 
using a RDBMS, an "article" is more than just the article table.  There 
are 5 tables that hold all the information for an article.  So, we 
wrapped all that into a single object that could be cached.

Now, if this does not meet your situation, you may want to look at 
something like MySQL Cluster.  For our HA, we have put most primary key 
data into MySQL Cluster and use memcached to cache large chunks of data 
made up from many data sources.

The other issue to take into consideration is connection numbers.  Unit 
testing can only take you as far as you can replicate a real world 
situation.  You may find that MySQL does not perform as well as 
memcached under real world load.

-- 

Brian Moon
Senior Developer
------------------------------
http://dealnews.com/
It's good to be cheap =)