garbage collector

[Miguel DeAvila]

On Friday 23 May 2008 15:40:36 Ivan Szanto wrote:
> But after a couple of days of usage we began to have problems. These 
> problems were helped by restarting memcached.
> After having a closer look at these problems, it appeared that the 
> memory was gradually getting full.
> This was kind of strange, because we use an expiration time of 6 hours, 
> which would mean that after a couple of days all items would have to be 
> evicted except those that were put in the last 6 hours.
> 
> We suspected that most of the expired items stay in the cache and never 
> get evicted, because the value of the STATS evictions was always 0 and 
> because we found a lot of items older than 6 hours.

This sounds like a reference count problem that dormando referred to later in the thread.

See 

	http://www.nabble.com/out-of-memory-errors-td16171089.html
	http://www.nabble.com/Loop-detection-in-assoc_find..-ts16775479.html




> 
> The only place I found in the code where evictions are taking place is 
> in do_item_alloc, which means that an item can only be evicted in the 
> case when there is the need for allocating a new item. This means there 
> is no separate garbage collection as such in memcached. This is good, 
> because that would need lots of CPU ticks to run, lowering the 
> performance of the cache.
> 
> We found that we need to evict expired items from the cache and 
> therefore we implemented an external garbage collector. We run this 
> program regularly and we found that this solves our problem. This means, 
> memory usage is constantly low now.
> 
> And once again:
> 
> We would like to make it available as an open source software. Is this 
> interesting to the members of this list?
> Please let me know where we can post it.
> 
> Kind regards,
> Ivan
>