memcached not using allocated memory
blewis at etrade.com
Fri Apr 20 15:03:01 UTC 2007
I'm going with the default 1.25 growth as it provides this range:
slab class 24: chunk size 14200 perslab 73
slab class 25: chunk size 17752 perslab 59
slab class 26: chunk size 22192 perslab 47
I assumed that the "bytes" stat would contain the count for allocation for each chunk(17752), and not the actual amount of the chunk used, but that might not be the case.
If the values are 16384 bytes, they should use the 17752 byte chunk and occupy 92% of it. I'm not sure about the space for the keys/flags but the keys are all between 10 and 15 bytes. I've tried tweaking the -f option and the default has provided the best result to date. Any suggestions. thanx.
From: Steven Grimm [mailto:sgrimm at facebook.com]
Sent: Fri 4/20/2007 12:28 AM
To: Marcus Bointon
Cc: Lewis, Bill; memcached at lists.danga.com
Subject: Re: memcached not using allocated memory
Marcus Bointon wrote:
> I'm not familiar with the internals, but I believe that memory chunks
> are allocated in power-of-2 sized chunks. 16394 is 10 bytes over
> 16384, which is 2^14, so your 16394 bytes will be stored in a 32768
> size chunk (2^15), hence the wastage. It's just unfortunate that your
> data is that size - it couldn't get much less efficient! That's my
> theory anyway, could be completely wrong of course!
That's somewhat plausible, although modern versions use a power of 1.25
to determine the chunk sizes.
It's easy to test that theory: there's a command-line option (-f) that
lets you set the factor that's used. Try reducing it to 1.05 or some
similarly small value and you should get a chunk size much closer to
your data size.
Also, if that's what's going on, and all the objects in the cache are
truly fixed-size, you can use another command line option to tune the
smallest chunk size. It is perfectly fine to set that to the exact value
needed to hold your fixed-size objects (remember that you need to leave
room for the key as well, plus some space for the flags value; should be
pretty easy to experimentally determine the optimal size.) Then there
will be no wasted memory in your slabs at all.
More information about the memcached