Replication

[Brian Beuning]

Hi,

I am looking into using memcached for our project.  We are not a big web
site
like many of your users.  Instead we are a "scale out" architecture for
financial
processing.

Our systems use lots of processes (think 100 or more) to crunch away at DB
jobs.
We currently cache something like 400 MB of data into each of these
processes.
I would like to move this 400 MB into memcached to reduce our memory
footprint.

We get a fixed time window to process thousands of jobs.  If the cached data
needs to
be gotten from the DB, we will blow out the time window and have unhappy
accountants.

One instance of memcached could handle our tiny 400 MB with no problem.
It can probably even handle the load of 100 processes hitting it.  But I am
concerned if memcached went down then we would miss our fixed time window.

Ideally we would like to have a few memcached instances each with a full
copy of the 400 MB.  The Wiki says memcached does not do replication.  

It seems to me, the client side could send all changes to all instances.
Of course the fun starts when an instance goes down, or the network
partitions.  (Someone mentioned the Amazon Dynamo paper.  It gives
lots of issues to consider.)

Has anyone tried replication with memcached?
What has been your experiences?

Thanks!
Brian Beuning