Transparent failover and restore?

[Timo Ewalds]

Brad Fitzpatrick wrote:

>On Sat, 18 Dec 2004, Jon Valvatne wrote:
>
>  
>
>>I think the point is this: If you're using memcached in a way that means
>>losing a server will bring down the system, then you should probably be
>>looking at using the MySQL "redundant storage thing" instead of the
>>memcached "cache thing". Traditional usage of a cache usually means
>>losing parts or all of it is *not* fatal.
>>    
>>
>
>Exactly.
>
>Or do what LiveJournal does:  have a dozen memcached machines, each with
>about the same amount of cache.  So if you lose a machine, you only lose
>1/12th of the cache, until it re-hashes and populates over the next 10
>minutes.
>
>I suppose I see the arguments from the people who only have 2 machines
>and want good cache hit rate performance in the face of failure, but the
>cache coherency problems just seem too hard to re-implement when solutions
>already exist.
>
>Perhaps memcached atop NDB cluster (what MySQL Cluster is built on)
>would be interesting.  NDB would do the cache coherency across the
>cluster, and memcached could simply do the protocol handling and LRU
>maintenance and cache discarding...
>
>- Brad
>
>
>  
>

If cache speed isn't all that much of an issue, why not just run the 
cache off mysql cluster. Create a table of the form: key, expiry, flags, 
data (or whatever), primary on key, and cluster it across however many 
machines you want. You then keep your main database on normal mysql 
servers so size isn't an issue. It is still in ram and everything is off 
the primary key with no joins, so lookups shouldn't be all that slow 
either (though certainly slower than memcached). I guess you could even 
build it into the memcached api to automatically put add/set/remove to 
the mysql cache, and check there if you get a cache miss in memcached or 
you detect a server down. Depending on your usage, you could probably 
even get away with doing that on a normal mysql server instead of the 
cluster. I certainly had one table specific for caching before I found 
memcached, and it worked great, though it was a bit slower, especially 
if you didn't combine cache hits (ie SELECT ... WHERE key IN 
('key1','key2') ).

Timo