Is there a limit on the # of objects in memcache

Brad Fitzpatrick
Mon, 12 Jul 2004 11:48:11 -0700 (PDT)

No limit.  We've had way more than that.

I imagine you've allocated all your possible 1MB pages, and now you have
tons of slabs of different sizes with free space, but you're trying to add
an item of a size class which was never allocated space.

This is hard for people to understand unless they understand slab caches
so we've been working on making this balancing all automatic, but in the
meantime you have to either restart your memcached once your
size-usage-patterns change, or run a script against your memcacheds to
reassign pages from one size class to another, based on the age depth of
each class.

For instance, if you have a dozen pages holding 64k objects and they're
living for 5 days, but then you have 2 pages holding 128k objects lasting
45 seconds, you should drop some of your 64k pages and make them do 128k

Confusing, huh?

Anatoly:  any progress on the "bulldozer" allocator?

I suppose I should include in the distro a Perl script to do the
auto-balancing, but I've never written it.... just used the command by
hand with telnet when we added the class-reassign support to memcached.

Any volunteers?  Basic outline:

-- connect to a memcached IP/port
-- look at all slab classes
-- if no memory availabe (all pages allocated),
-- see if there's a big disparity between class ages
-- run the reassign page command (or whatever it is,
   see source)

- Brad

On Mon, 12 Jul 2004, J. Scott Johnson wrote:

> Hi there,
> Quick memcache question if you have a sec.  We're experimenting with some
> heavy memcache usage and we're seeing that even though there is more
> memory available, once we create 2,000,000+ keys we get a memcache client
> error out of memory.
> Any thoughts?
> Thanks
> Scott