Cache data that whose size changes over time

[Håkan Waara]

Hi dormando, thanks for your reply.

Thanks for confirming that the issues we thought we were seeing should 
not have happened. It actually helped us to look for other possible 
causes... :-)

It turns out that one of the memcached processes in our web server 
cluster was running an old version. Whenever we tried to save a specific 
piece of data, it was redirected there, and we got the "out of memory" 
error. Now we've upgraded it as well, and everything seems to work just 
fine.

Thanks!

Håkan Waara
Travelocity Nordic

dormando wrote:
> Hey,
>
> Are you running memcached with the -M option? What version are you 
> using? What clients?
>
>> The thing is, we don't have 100% control over the size of the data we 
>> typically cache. It may change over time. The problem then of course 
>> becomes that the slab classes are already populated, so things of 
>> other sizes won't find a slab class that matches their size, and thus 
>> cannot be cached at all.
>
> This _should_ only happen when running in -M mode. What should 
> otherwise happen are cache evictions. So if there isn't enough memory 
> available in a slab class it would eject something else from the cache 
> first.
>
>> One common suggestion seems to be to "warm up" the cache. Would it be 
>> a bad idea to warm up every slab class with a minimum of 1 item, just 
>> to make sure they're all "open"? I guess that is something we'd like 
>> to avoid.
>
> Recent releases of memcached do this already, expressly to avoid that 
> issue.
>
>> Another way is to manually adjust the slab sizes over time, using 
>> memcached-tool to move items around, or even restarting the daemons. 
>> This seems a bit tedious though.
>
> It might not work very well. The slab reassignment stuff is aligned 
> for a factor of 2.0, but the default slab class factor is now 1.2... I 
> don't fully grok how that affects everything yet, but noticed while 
> reading the code.
>
>> Is there any way to automatically adjust the slabs over time, as the 
>> data stored changes? What's the current practice for dealing with 
>> this problem, for other users of memcached?
>>
>> Any advice appreciated - thanks!
>
> No way as of yet, but you probably shouldn't be getting out of memory 
> errors outright, so there's a chance something else is holding you 
> back here.
>
> -Dormando