file side memcached

[rm_memcached at cheapcomplexdevices.com]

Brian Moon wrote:
>> Why not just run memcached with a larger memory size than physical 
>> memory and let the kernel's swaping logic handle which blocks and when 
>> get swapped out?
> 
> That would mean that every other process on the server was getting 
> swapped out.  

Nah.   In both BDB and Memcached's case the most frequently
accessed pages in the system will be in memory, and the least
frequently will be on disk.   In BDB, the frequently accessed
pages will be in the kernel's page caches and up-to-date
versions will not be on the disk anyway (unless you send a
bunch of unnecessary sync() calls).  And it's true with
Memcached, where the least frequently used pages should
be swapped to disk if there are better uses for the memory.

 > Swap is for when something is happening that is NOT
> expected.  

Totally disagree.   Swap is useful for infrequently accessed
memory so that your valuable physical memory can be used for
frequently accessed data, whether those pages are backed
by files (bdb databases) or not.

Perhaps one of the most authoritative sources would
be Andrew Morton (linux 2.6 kernel maintainer)
who words it better than I can:

http://kerneltrap.org/node/3000
    "My point is that decreasing the tendency of the
    kernel to swap stuff out is wrong. You really
    don't want hundreds of megabytes of BloatyApp's
    untouched memory floating about in the machine.
    Get it out on the disk, use the memory for
    something useful."
    - Andrew Morton

 > Never rely on it.

Unless you're seeting /proc/sys/vm/swappiness to zero,
you *are* relying on swap to free up rarely used memory
to make it available for frequently used disk pages.
(though I can't speak for other OS's)