SERVER_ERROR out of memory on small puts to memcache.
dormando at rydia.net
Tue Nov 13 22:42:22 UTC 2007
There's 1M worth of slabs pre-allocated for each slab class, so that
Part of the issue is that error is the same for about 10-15 different
OOM conditions. Which is something I'll submit a patch to the list about.
Matt Knox wrote:
> Yeah, this was my impression as well. I wrote a small test-script in
> ruby to fill the cache with a bunch of objects of the same size, and
> then try to put in a small object, but it still did not reproduce the
> problem. I don't have a spare 3G of ram box to try it on.
> What happens if you try to store an object of N bytes, but there are no
> slabs with buckets of that size? I'd assume that you'd either put it in
> a bigger bucket or retire an entire slab.
> On Nov 13, 2007 5:25 PM, Jeremy Blain <jeremy at belent.com
> <mailto:jeremy at belent.com>> wrote:
> I've seen this happen in the past as well, but forgot about it till now.
> Bouncing the server also fixed it.
> My not very thorough looking about seemed to suggest the cache had
> allocated all it's slabs already to certain sized objects, and it
> to store the current object in a slab that had chunks of sizes that
> didn't exist.
> Matt Knox wrote:
> > The problem seems to have gone away on bouncing memcache, and we
> > replicate it, even though we started memcache in exactly the same
> > on the same machines. We'll probably make 3 1Gb instances instead of
> > the 3G instances if it recurs.
> > thanks for the help!
> > matt
> > On Nov 11, 2007 9:06 PM, dormando <dormando at rydia.net
> <mailto:dormando at rydia.net>
> > <mailto: dormando at rydia.net <mailto:dormando at rydia.net>>> wrote:
> > *pulls old mail out of garbage*
> > Curious if you're still having this issue? If it's been
> fixed/etc? I
> > didn't see any noise on the list about it since.
> > Matt Knox wrote:
> > > I consistently get an out of memory error when performing < 44
> > byte
> > > (key+data) puts to memcache, but when 'put'-ing the same key
> > with data
> > > that is long enough to exceed 44 chars key + data , I
> succeed. I
> > > observe this behavior using both the ruby client
> > (memcache-client 1.5)
> > > and the python one (python-memcached), although the python
> > client seems
> > > to break at 59 chars, rather than 44.
> > Spiffy! Can't reproduce it on a 64-bit host. Haven't tried
> > > I'm running memcached 1.2.2 with the following options:
> > >
> > > memcached -d -p 11211 -u nobody -c 1024 -m 3072
> > >
> > > The offending servers are recent CentOS running 3G of cache
> each on
> > > 32-bit boxes with 4G ram-if it matters, I can get details about
> > these.
> > I suspect that's a big deal. The maxbytes limit doesn't
> include random
> > buffers that are used for connections, stats commands, this and
> > that. So
> > you'll be apt to bowl over the 32-bit address space. I've never
> > trusted
> > that the different splits even work.
> > So, if you are still having this issue:
> > - How'd you build memcached?
> > - Exactly what version of centos?
> > - Does it happen after memcached has been up for a long time, or
> > immediately?
> > - Does it still happen if you lower the -m option to 1.6-1.8
> > gigabytes?
> > Thanks,
> > -Dormando
> (def (eval e l d c)
> (if (atom? e)
> ((ahandler (type e)) e l d c)
> (eval (car e) l d
> (fun (x)
> (evapp x (cdr e) l d c)))))
More information about the memcached