Strange memcached behavior on stress test
woodbri at swoodbridge.com
Sun Jun 11 22:45:18 UTC 2006
>> Debian 32bit Sarge with a HUGEMEM kernel patch and 6 GB of memory and
>> 1 GB configured for memcached. I grew to about 1.5 GB then started
>> throwing the errors.
> What were all of the options you used to start memcached?
/usr/bin/memcached -m 1024 -p 11211 -u root -l 10.14.12.101
>> The stress test script generated a random sized data packet (1-20KB in
>> size) then does a set and an immediate get and compares the set and
>> and got value, it counts it as an error if they do not match. this was
>> done via a perl script. Sorry, I don't have more specific data, I
>> would have to rerun the tests if I can get the server.
> Someone else can probably figure it out from this point, but I can't :(
> I saw in your test script that you were just counting failues, but not
> checking the type. Your test script will have to actually inspect the
> response of the set command and the response of the get command and log
> any actual errors, as well as anything logged on the server end (run
> memcached in the foreground for your tests).
I would happy take some changes to me script to do better diagnostic
checking and reporting if you have any specific ideas. I can check to
see if the set failed or if the get failed or succeeded but with
different contents. Is here more I can do?
> In my mind it could still be a few different things, but more
> information would narrow that down more. I'd also say give the facebook
> branch a try just in case that fixes it.
> However even without the facebook patches the memcached instances I run
> hav been chugging along for 6+ months without causing problems, nor
> needing restarts. We might not be pushing the memory limits as hard as
> your test script though.
I may need to wait for my sysadmin to get back from his honeymoon,
before I can get patches integrated and deployed to a server for
More information about the memcached