Patch: A different fix for the "out of memory" problem
sgrimm at facebook.com
Tue May 2 21:33:21 UTC 2006
The 1.1.13 prerelease has code to automatically preallocate a slab in
each slab class to avoid spurious "out of memory" errors when some slab
classes aren't in use by the time memcached reaches its memory limit.
Among other changes we've made to memcached 1.1.12 (patches
forthcoming!) we decided to solve the same problem in a different way:
if needed, we let memcached exceed its configured memory limit so it can
allocate at least one slab in each slab class. This is only on an
as-needed basis; if all slab classes are already in use by the time the
memory limit is hit, this doesn't eat any extra memory.
The disadvantage is that memcached no longer adheres as strictly to its
memory limit. The advantage is that if you don't ever need a slab of a
given class, you don't waste memory preallocating it. A lot of our
memcached servers are dedicated to fixed-size items, so we only ever
need a single slab class on those machines -- there's no point setting
aside megabytes of memory for "set" requests that can never happen.
We don't view the disadvantage as a big deal, in part because
memcached's memory usage isn't strictly lower than the configured limit
to begin with: additional memory is already used by the stack, allocated
memory other than the slabs, TCP buffers and so forth.
Anyway, here's the patch, which is about a tenth the size of the
--- ../memcached-1.1.12/slabs.c 2006-03-24 23:17:18.000000000 -0800
+++ slabs.c 2006-05-02 14:16:08.000000000 -0700
@@ -103,7 +103,7 @@
int len = POWER_BLOCK;
- if (mem_limit && mem_malloced + len > mem_limit)
+ if (mem_limit && mem_malloced + len > mem_limit && p->slabs > 0)
if (! grow_slab_list(id)) return 0;
More information about the memcached