Memcached implementation inquiry
brad at danga.com
Thu Apr 19 17:18:03 UTC 2007
Latency from round-trips is killing you, it's not memcached.
Just ping your memcached server and look at the ping round-trip time, then
multiply by 600,000.
$ ping 192.168.64.1
PING 192.168.64.1 (192.168.64.1) 56(84) bytes of data.
64 bytes from 192.168.64.1: icmp_seq=1 ttl=255 time=0.398 ms
64 bytes from 192.168.64.1: icmp_seq=2 ttl=255 time=0.407 ms
64 bytes from 192.168.64.1: icmp_seq=3 ttl=255 time=0.414 ms
64 bytes from 192.168.64.1: icmp_seq=4 ttl=255 time=0.400 ms
--- 192.168.64.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 2997ms
rtt min/avg/max/mdev = 0.398/0.404/0.414/0.025 ms
0.403 milliseconds * 600,000 = 4.03 minutes
It's not memcached that's slow... it's your huge loop with a network
round-trip in it.
On Thu, 19 Apr 2007, Michael Firsikov wrote:
> We have recently stumbled upon Memcached after our mysql databases were
> unable to withstand the concurrency load (even in replicated environment)
> I have thoroughly read almost all archived digests; however, I still have
> not been able to properly grasp one important concept. (Do not worry, I will
> not ask whether you can list all stored keys :)
> We have a large subscriber database (1.4 million users roughly), each having
> somewhat detailed profile. For benchmark tests I am preloading the
> information into memcached, and when pulling info for a particular profile,
> everything is pretty smooth. (single get for a specific key)
> However, one of the main reasons to explore memcache for us, were the
> searches. (The complexity of searches in MySQL (a myriad of joins, etc)
> resulted in sub-par performance). I am pretty certain it is against
> memcached best practices, but I have done a basic loop to get thru roughly
> 600K records, get and check a value.
> This process takes over 2 minutes on a decent box running Ubuntu server with
> 2GB ram allocated to memcached. Is it bottlenecking at TCP level of the
> connection (roughly would need 2MB transfer for the 600,000 gets)?
> Or is the retrieving memory key takes the bulk of the time?
> Thank you in advance for reading this convoluted message :)
> P.S. Setup is 1 memcached machine, with everything running in local, nothing
> put apache+php, (pecl extension for memcached), dedicated machine for the
> test, memcached never hits swap as it only takes 500MB for the 600K records.
> Michael F
More information about the memcached