Cache data that whose size changes over time

[dormando]

Hey,

Are you running memcached with the -M option? What version are you 
using? What clients?

> The thing is, we don't have 100% control over the size of the data we 
> typically cache. It may change over time. The problem then of course 
> becomes that the slab classes are already populated, so things of other 
> sizes won't find a slab class that matches their size, and thus cannot 
> be cached at all.

This _should_ only happen when running in -M mode. What should otherwise 
happen are cache evictions. So if there isn't enough memory available in 
a slab class it would eject something else from the cache first.

> One common suggestion seems to be to "warm up" the cache. Would it be a 
> bad idea to warm up every slab class with a minimum of 1 item, just to 
> make sure they're all "open"? I guess that is something we'd like to avoid.

Recent releases of memcached do this already, expressly to avoid that issue.

> Another way is to manually adjust the slab sizes over time, using 
> memcached-tool to move items around, or even restarting the daemons. 
> This seems a bit tedious though.

It might not work very well. The slab reassignment stuff is aligned for 
a factor of 2.0, but the default slab class factor is now 1.2... I don't 
fully grok how that affects everything yet, but noticed while reading 
the code.

> Is there any way to automatically adjust the slabs over time, as the 
> data stored changes? What's the current practice for dealing with this 
> problem, for other users of memcached?
> 
> Any advice appreciated - thanks!

No way as of yet, but you probably shouldn't be getting out of memory 
errors outright, so there's a chance something else is holding you back 
here.

-Dormando