garbage collector

[Ivan Szanto]

Hi,

we wrote an external garbage collector for memcached and it works fine.
We would like to make it available as an open source software. Is this 
interesting to the members of this list?
Please let me know where we can post it.

The story:

We apply memcached to cache all kinds of items of a largish website 
written in php.
We found it's working great!

But after a couple of days of usage we began to have problems. These 
problems were helped by restarting memcached.
After having a closer look at these problems, it appeared that the 
memory was gradually getting full.
This was kind of strange, because we use an expiration time of 6 hours, 
which would mean that after a couple of days all items would have to be 
evicted except those that were put in the last 6 hours.

We suspected that most of the expired items stay in the cache and never 
get evicted, because the value of the STATS evictions was always 0 and 
because we found a lot of items older than 6 hours.

The only place I found in the code where evictions are taking place is 
in do_item_alloc, which means that an item can only be evicted in the 
case when there is the need for allocating a new item. This means there 
is no separate garbage collection as such in memcached. This is good, 
because that would need lots of CPU ticks to run, lowering the 
performance of the cache.

We found that we need to evict expired items from the cache and 
therefore we implemented an external garbage collector. We run this 
program regularly and we found that this solves our problem. This means, 
memory usage is constantly low now.

And once again:

We would like to make it available as an open source software. Is this 
interesting to the members of this list?
Please let me know where we can post it.

Kind regards,
Ivan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/memcached/attachments/20080524/8948fc10/attachment.htm