brad at danga.com
Mon Jan 8 19:04:17 UTC 2007
On Mon, 8 Jan 2007, Steven Grimm wrote:
> Philip Neustrom wrote:
> > Steve mentions that with tens of millions of items in a cache, that
> > the cost of having an additional byte is too great. If I'm not
> > mistaken, this is only 10MB of extra memory used, and in this
> > situation (10 million items), you're looking at a cache which is
> > likely in the 100's of GBs. This doesn't seem like a measurable
> > difference to me (less than .01 percent?)
> That assumes all your objects are large. We store a lot of very short
> values in our cache, just a few bytes in many cases. On some of our
> memcached instances, we often see upwards of 40 million entries (that's
> per instance, not total across all instances.) You're correct that
> that's less than a percent of the cache, but it's closer to .5% than
> .01%, easily in the "measurable" range. I can say that with certainty
> since we've actually measured the difference as we've made various
> changes to the item structure over time.
> If someone changes memcached to use a 2-byte key length, I suppose I
> won't object too much -- .5% is significant but not deadly -- but we
> certainly have zero use for such a change here. Just make sure you do it
> in such a way that byte alignment requirements don't cause you to add
> padding to the item structure, especially on 64-bit systems; .5% is not
> too bad but 4% (worst case, if you end up adding 8 bytes per item)
> certainly would be. And of course obviously Brad gets the final say!
Struct alignment is all I care about too. Not against a longer limit.
I worry about the clients a bit, though.
More information about the memcached