Memcached keys expiring to soon

[Garth Webb]

Your hit rate is really low, %53.  This seems to indicate that the LRU
is turning over many of your keys.  It might be that memory isn't being
used as you think.  Have you run the memcached-tool that comes with the
memcached source?  It will show you how the slabs are being utilized.

Garth

On Wed, 2006-12-20 at 20:23 +0100, Kenneth Pedersen wrote:
> Hi,
> 
> We have a cluster with five webservers accessing a single memcached
> instance. When setting keys, the keys typically have a lifetime of only
> a few seconds (typically around 10s) in the cache, even if we set expiry
> time to 0 (do not expire). The memory usage of memcached is not even
> near its maximum quota. We have also tested explicitly setting expiry
> time to some large value (e.g. 10000 seconds), but this makes no difference.
> 
> For example, this behaviour is present when memcache is using 80 MB out
> of a maximum quota of 256. The machine the memcache daemon is running on
> has over 500 MB free memory.
> 
> We are using version 1.2.0 of memcached, and this is the command line
> used for starting the daemon:
> 
> /usr/bin/memcached -d -p 11211 -l 10.1.2.3 -m 256 -c 1024 -u nobody
> 
> I have made a small test program that exposes this behaviour, it is
> attached below. Note that this only happens when the webserver daemons
> are actively using the cache, when we run the script with another
> memcached instance that has no other clients, the keys persist in the
> cache as expected.
> 
> Does anyone have an idea about what is going on here?
> 
> Regards,
> Kenneth Pedersen
> Netlife Backup Solutions AS
> 
> Below:
> test script (uses python memcache bindings)
> typical output from "stat" command
> output from "free"
> 
> -------------
> import memcache
> import time
> import sys
> 
> mc = memcache.Client(['10.1.2.3'])
> 
> print "Setting a value"
> mc.set('foo', 'bar')
> t = time.time()
> 
> print "Waiting for value to expire from cache",
> sys.stdout.flush()
> while True:
>     if mc.get('foo') is None: break
>     print ".",
>     sys.stdout.flush()
>     time.sleep(1)
> print ""
> print "Item expired in %.2f seconds." % (time.time() - t)
> 
> -------------
> STAT pid 15782
> STAT uptime 410
> STAT time 1166641383
> STAT version 1.2.0
> STAT pointer_size 32
> STAT rusage_user 6.736421
> STAT rusage_system 14.232889
> STAT curr_items 66637
> STAT total_items 202056
> STAT bytes 30107093
> STAT curr_connections 82
> STAT total_connections 90607
> STAT connection_structures 227
> STAT cmd_get 108548
> STAT cmd_set 202056
> STAT get_hits 57672
> STAT get_misses 50876
> STAT bytes_read 115376234
> STAT bytes_written 31361237
> STAT limit_maxbytes 268435456
> END
> 
> -------------
>              total       used       free     shared    buffers     cached
> Mem:        906340     852888      53452          0     237860     205828
> -/+ buffers/cache:     409200     497140
> Swap:      1052248        112    1052136
> 
>