Large memory support
Wed, 18 Feb 2004 21:41:35 -0800
Actually, this was on RedHat Enterprise Linux 64 on a dual Opteron =
system. No matter what I entered as a memory size, it listed 2GB as =
maxbytes in STATS. When I added more records after it reached 1.9GB it =
would start tossing things.
When I replaced the unsigned ints with unsigned long longs, it would =
grow past 2GB.
Seems like switching to size_t would be the best path. What do you =
p.s. - Our goal is to have a ~5GB cache with an entire database. We =
want to disable expiration (i.e. nothing getting removed out of the =
cache) and have the cache be the definitive read only source. It would =
have a syncing process to keep it up to date with any changes.
From: Brad Fitzpatrick [mailto:email@example.com]
Sent: Wed 2/18/2004 9:37 PM
To: Jason Titus
Subject: Re: Large memory support
I've built and run memcached on 64-bit machines.
On a 32-bit machine you won't be able to store more memory than your
operating system is giving you address space for. (Not sure if
your OS is giving you a 2G/2G, 3G/1G, or 3.5G/0.5G split)
I assume you're on 32-bit? Otherwise, isn't an "unsigned long" just a
uint64 on 64-bi machines? Or should we be using void* somewhere we're
using unsigned int?
Let me know your OS, arch, and the contents of "stats maps" (if on =
$ telnet localhost 11211
Connected to localhost.
Escape character is '^]'.
08048000-08050000 r-xp 00000000 03:03 2167117 =
08050000-08051000 rw-p 00008000 03:03 2167117 =
08051000-09c02000 rwxp 00000000 00:00 0
40000000-40011000 r-xp 00000000 03:03 165006 /lib/ld-2.3.1.so
40011000-40012000 rw-p 00011000 03:03 165006 /lib/ld-2.3.1.so
40017000-4011f000 r-xp 00000000 03:03 165009 /lib/libc-2.3.1.so
4011f000-40125000 rw-p 00107000 03:03 165009 /lib/libc-2.3.1.so
40125000-40128000 rw-p 00000000 00:00 0
40128000-40131000 r-xp 00000000 03:03 165014 =
40131000-40132000 rw-p 00009000 03:03 165014 =
40132000-40142000 r-xp 00000000 03:03 165013 /lib/libnsl-2.3.1.so
40142000-40143000 rw-p 00010000 03:03 165013 /lib/libnsl-2.3.1.so
40143000-80946000 rw-p 00000000 00:00 0
bfffc000-c0000000 rwxp ffffd000 00:00 0
On Wed, 18 Feb 2004, Jason Titus wrote:
> Looks like we just stumbled onto some 32 bit limits in memcached. =
> wondering how you guys want to deal with supporting >2GB per cache. =
> have experimented with switching some of the unsigned ints into =
> long longs and it seemed to work (>4GB seemed ok. No thorough testing
> yet though). A cleaner solution might be adjusting the memory related
> variables to size_t or some such so that they work well on 32 and 64 =
> What makes the most sense to you folks?