Memcache not freeing memory when a key expires

Evan Fribourg evan at deniromarketing.com
Tue Feb 28 19:05:02 UTC 2006


Under our unit testing with memcached, when we insert an item with a TTL
greater than 0, when the item expires the item is not removed from memory
until a GET command is issued on the key.  Based on the lazy reaper, this is
normal operating procedure. In production, however, our memcached server
will max out its usable space and NOT seem to drop keys based on the LRU
scheme, with settings.evict_to_free == 1.

The keys that we use in our server are mostly unique -- they tend be to be
re-used for a little while, and then not used ever again.  Because there was
no reaper to go through and delete these expired keys, the memcached server
kept growing and growing until we reached a point where memcache was unable
to evict any items.  Our implementation of a reaper command within memcached
collects our garbage and enables us to not have to restart memcached every
few days.

Evan Fribourg
 

-----Original Message-----
From: memcached-bounces at lists.danga.com
[mailto:memcached-bounces at lists.danga.com] On Behalf Of Jason Rimmer
Sent: Tuesday, February 28, 2006 7:18 AM
To: memcached at lists.danga.com
Subject: Re: Memcache not freeing memory when a key expires

	I'm on the list digest so my apologies if this has already been
addressed:
	Why would you do this; what purpose would the reaper serve?  As Brad
mentioned previously memcached is an LRU cache with essentially a lazy
'reaper' to maintain O(1).  In general an active reaper would have two
purposes: 1) Remove expired entries to make room for more and 2) Ensure that
expired entries are no longer accessible.  Brad addressed both points in
that for the first case expired entries are automatically removed via LRU
when memcached comes under memory pressure.  With the second case memcached
ensures that expired entries are not retrievable (the fact that those
entries happen to be removed or 'reaped' when accessed is orthogonal to the
retrieval attempt).
	What are you trying to accomplish with your active reaper and how
will the reaper know that a key is expired and hence appropriate for
'removal'?



More information about the memcached mailing list