Thots on memcache redundancy
lists at benrey.is-a-geek.net
Fri Oct 13 07:22:33 UTC 2006
Two thoughts on memcache redundancy.
If you have an active-passive failover with heartbeat, you can write a
very simple memcache script on the passive node to poll the active
node's memcache and keep a hot copy of it. This is the simplest
replication you can do.
Another way, if you want to manage it at the application level, is to
have a side channel to administrate availability of memcache
repositories by managing access to memcache keys through the memcache
client. You would do this by marking a memcache server as "deactivating"
and each time an entry in the deactivating cache is requested, it either
gets moved to another memcache server or regenerated. When you have an
allowable threshold of "transitioned" keys in your deactivating memcache
server (such as would not spike your database), you just purge the rest
of the keys by removing the reference to the deactivated server. I think
some memcache clients hash your keys across an array of servers, so
removing a server means you have to adapt that behavior in the memcache
where the old server location would likely be
$serverindex = intval( $key ) % count( $oldservers_list );
and if that key exists, pull it and rehash it to a new hash entry:
$serverindex = intval( $key ) % count( $newservers_list );
You could keep a count of how many keys have been mapped to a server
entry, and during the transition, you could create a secondary count of
moves. Since you probably don't want to manage a list of keys and
duplicate the logic of when they expire, probably you'd just have to
guess what ratio to kill it off at.
Or, your memcache client would have to go thru and rehash the mapping
from the deactivating server to the remaining servers, moving every
entry. This might have to be done such that you don't over-fill adjacent
machines and force out too much of their entries, and don't incur undo
load doing a lot of really fast transfers.
More information about the memcached