Strange memcached behavior on stress test
dormando at rydia.net
Sun Jun 11 19:25:21 UTC 2006
> Debian 32bit Sarge with a HUGEMEM kernel patch and 6 GB of memory and 1
> GB configured for memcached. I grew to about 1.5 GB then started
> throwing the errors.
What were all of the options you used to start memcached?
> The stress test script generated a random sized data packet (1-20KB in
> size) then does a set and an immediate get and compares the set and and
> got value, it counts it as an error if they do not match. this was done
> via a perl script. Sorry, I don't have more specific data, I would have
> to rerun the tests if I can get the server.
Someone else can probably figure it out from this point, but I can't :(
I saw in your test script that you were just counting failues, but not
checking the type. Your test script will have to actually inspect the
response of the set command and the response of the get command and log
any actual errors, as well as anything logged on the server end (run
memcached in the foreground for your tests).
In my mind it could still be a few different things, but more
information would narrow that down more. I'd also say give the facebook
branch a try just in case that fixes it.
However even without the facebook patches the memcached instances I run
hav been chugging along for 6+ months without causing problems, nor
needing restarts. We might not be pushing the memory limits as hard as
your test script though.
More information about the memcached