Transparent failover and restore?

[Andreas Vierengel]

Am Mo, den 20.12.2004 schrieb Kevin A. Burton um 0:56:
> Brad Fitzpatrick wrote:
> 
> >Guys,
> >
> >If you want to use memcached as a data store instead of a /cache/, then
> >use MySQL Cluster:
> >
> >   http://www.mysql.com/cluster/
> >
> >It was designed for that, doing the whole redundant storage and two-phase
> >commit thing, while memcached was designed to be a cache.
> >  
> >
> I've also thought of using MySQLs memory tables as an LRU cache as a 
> memcached replacement. It would support btree indexing, hash indexing, 
> replicated data, etc. The client would have to route storage of the keys 
> via hash() % Nservers but this isn't hard. The big downside is that (at 
> least in Java) the MySQL JDBC driver is about 1ms where it seems that an 
> optimized memcached could perform 500 gets() per 1ms.
> 
> Of course that argument is negated a bit if the mysql driver could be 
> made more efficient.
> 
> Theres also the added benefit that (maybe) MySQL would have better 
> memory storage since it knows the exact object size due to schema and 
> coudl serialize correctly espcially if you use an OR layer. Though my 
> recent patches to java-memcached make it use a LOT less memory so this 
> might be irrelevant.
> 
> Kevin

hmm. mysql uses a thread per connection, which preallocates memory from
the thread-stack (allocation is fragmented) and also pre-allocates
memory for internal things like sort-buffer and so on. So in my opinion
it would consume too much memory for handling the connection and so on.
You have to use more hardware to achieve the same with memcached.

Replication in standard mysql is asynchron whereas replication in
Mysql-cluster is synchron. Synchron is always slower. But asynchron has
the disadvantage that a key/value differs for "replication-time" which
is sometimes unacceptable.

Further, according to my experience, asynchronous request-handling in
one process with epoll() or kqueue() is still faster than
context-switches between thousand threads.

I simply have thougth about having a failover-machine for several
memcached-server which is feeded from a client that is listening on a
multicast-udp-socket. The "normal" clients send their modify-requests
additionally via udp to the multicast-address if the value is <64k (
>99% of my values are beyond this). This should not hurt performance and
this will add me enough reliability, if a memcached-server dies.

--Andy