max memory allocation

Anatoly Vorobey mellon@pobox.com
Fri, 9 Apr 2004 05:37:39 +0300


On Thu, Apr 08, 2004 at 06:15:16PM -0700, Brad Fitzpatrick wrote:
> Crap.
> 
> At least Linux and OSX got ~2GB.
> 
> But that's with malloc.  We should do another test with mmap.

We could allocate in chunks of 500Mb (unless given less memory initially 
of course), and just ignore the boundaries (when we need to store an 
item crossing the boundary, just waste the remaining space till the 
boundary and start at the next chunk). Then e.g. for a 2Gb instance of 
memcached and largest item size of 1Mb we'd never lose more than 4Mb
on this, which is acceptable.

The reason I really don't want items being non-contiguous is that it 
complicated working with keys (insanely), and sending items out to the 
socket (considerably).

-- 
avva