garbage collector

[dormando]

Hey,

Unfortunately you just patched the wrong problem. Memcached does not
need an active garbage collector.

Are you sure your machine was not swapping? Did you set the memory limit
too high? How did you verify it was "slow" ? What symptoms were you seeing?

"evictions" counters only go up when a *valid* item is removed from
cache ahead of time. Your cache was never quite full enough to
preemptively expire valid items.

Memcached is designed to use up to the limit of memory you specify, then
it will re-use that memory in a very efficient way, so it will never
release memory back to the operating system. If you're hitting swap, set
the maxbytes value to be lower with the "-m" option.

-Dormando

Ivan Szanto wrote:
> Hi,
> 
> we wrote an external garbage collector for memcached and it works fine.
> We would like to make it available as an open source software. Is this
> interesting to the members of this list?
> Please let me know where we can post it.
> 
> The story:
> 
> We apply memcached to cache all kinds of items of a largish website
> written in php.
> We found it's working great!
> 
> But after a couple of days of usage we began to have problems. These
> problems were helped by restarting memcached.
> After having a closer look at these problems, it appeared that the
> memory was gradually getting full.
> This was kind of strange, because we use an expiration time of 6 hours,
> which would mean that after a couple of days all items would have to be
> evicted except those that were put in the last 6 hours.
> 
> We suspected that most of the expired items stay in the cache and never
> get evicted, because the value of the STATS evictions was always 0 and
> because we found a lot of items older than 6 hours.
> 
> The only place I found in the code where evictions are taking place is
> in do_item_alloc, which means that an item can only be evicted in the
> case when there is the need for allocating a new item. This means there
> is no separate garbage collection as such in memcached. This is good,
> because that would need lots of CPU ticks to run, lowering the
> performance of the cache.
> 
> We found that we need to evict expired items from the cache and
> therefore we implemented an external garbage collector. We run this
> program regularly and we found that this solves our problem. This means,
> memory usage is constantly low now.
> 
> And once again:
> 
> We would like to make it available as an open source software. Is this
> interesting to the members of this list?
> Please let me know where we can post it.
> 
> Kind regards,
> Ivan