Memcached implementation inquiry
michael at vstadi.com
Thu Apr 19 16:41:26 UTC 2007
We have recently stumbled upon Memcached after our mysql databases were
unable to withstand the concurrency load (even in replicated environment)
I have thoroughly read almost all archived digests; however, I still have
not been able to properly grasp one important concept. (Do not worry, I will
not ask whether you can list all stored keys :)
We have a large subscriber database (1.4 million users roughly), each having
somewhat detailed profile. For benchmark tests I am preloading the
information into memcached, and when pulling info for a particular profile,
everything is pretty smooth. (single get for a specific key)
However, one of the main reasons to explore memcache for us, were the
searches. (The complexity of searches in MySQL (a myriad of joins, etc)
resulted in sub-par performance). I am pretty certain it is against
memcached best practices, but I have done a basic loop to get thru roughly
600K records, get and check a value.
This process takes over 2 minutes on a decent box running Ubuntu server with
2GB ram allocated to memcached. Is it bottlenecking at TCP level of the
connection (roughly would need 2MB transfer for the 600,000 gets)?
Or is the retrieving memory key takes the bulk of the time?
Thank you in advance for reading this convoluted message :)
P.S. Setup is 1 memcached machine, with everything running in local, nothing
put apache+php, (pecl extension for memcached), dedicated machine for the
test, memcached never hits swap as it only takes 500MB for the 600K records.
More information about the memcached