<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META content="MSHTML 6.00.6000.16414" name=GENERATOR></HEAD>
<BODY>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>Here
is what I am thinking. Please poke holes in this.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>We
currently have 100 processes each loading 400MB which is 40GB of
RAM.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>What
if we set up 5 independent memcached servers (not one
cache</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>with 5 machines, but 5 caches each with a
full copy of the data)</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>and
we spread the 100 processes across 5 caches. One
process</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>only
talks with one cache, and each cache is serving 20
processes.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>If one
cache fails, the 20 processes using that cache switch to the
other</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>4 caches. When the dead </SPAN></FONT><FONT
face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>cache comes back,
the 20 processes switch</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>back
to it and start reloading </SPAN></FONT><FONT face=Arial color=#0000ff
size=2><SPAN class=953282514-24102007>data naturally. (If we can arrange
that the</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>clients move back at different times, maybe the storm
can be reduced.)</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>This
sounds safe to me.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>A more
advanced plan might have one process check 2 caches for
data.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>If a
key is found in one cache but not the other, then it stores the
key-value</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>in the
other cache. (In a failure scenario, this would let us load
</SPAN></FONT><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>a </SPAN></FONT><FONT face=Arial color=#0000ff
size=2><SPAN class=953282514-24102007>freshly</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>recovered cache from other caches instead of the
DB.) If the </SPAN></FONT><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>key </SPAN></FONT><FONT face=Arial color=#0000ff
size=2><SPAN class=953282514-24102007>is not</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>in
either cache, then it hits the DB </SPAN></FONT><FONT face=Arial color=#0000ff
size=2><SPAN class=953282514-24102007>and saves the key-value in
</SPAN></FONT><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>both </SPAN></FONT><FONT face=Arial color=#0000ff
size=2><SPAN class=953282514-24102007>caches.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>This
feels like it is moving </SPAN></FONT><FONT face=Arial color=#0000ff
size=2><SPAN class=953282514-24102007>into the hard replication issues
space.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>Memcached as it is today provides a certain good level
of reliability</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>(with
blazing performance). </SPAN></FONT><FONT face=Arial color=#0000ff
size=2><SPAN class=953282514-24102007>Some situations are going to require
more</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>reliability, and one way </SPAN></FONT><FONT
face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>to get that is
replication. If there is another way</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>to get
more reliability </SPAN></FONT><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>I am very interested in hearing about
it.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>MySQL
without ACID transactions supports a certain level of
reliability.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>Some
situations using MySQL need more reliability so MySQL added</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN class=953282514-24102007>the
</SPAN></FONT><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>ACID backend.</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007></SPAN></FONT> </DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007>Brian</SPAN></FONT></DIV>
<DIV><FONT face=Arial color=#0000ff size=2><SPAN
class=953282514-24102007></SPAN></FONT> </DIV>
<BLOCKQUOTE>
<DIV class=OutlookMessageHeader dir=ltr align=left><FONT face=Tahoma
size=2>-----Original Message-----<BR><B>From:</B> Clint Webb
[mailto:webb.clint@gmail.com]<BR><B>Sent:</B> Tuesday, October 23, 2007 9:38
PM<BR><B>To:</B> Marcus Bointon<BR><B>Cc:</B> Brian Beuning; Memcached
(E-mail)<BR><B>Subject:</B> Re: Replication<BR><BR></FONT></DIV>The problem
with a replicated cache is figuring out what to do if one fails.
Memcached effectively solves this problem by not doing replication. I
strongly agree with this approach unless you have a VERY good reason not to,
and in that case, memcached is probably not a very good choice. <BR><BR>What I
recommend is using multiple memcached caches, even if all your cached data can
fit in one instance, spread it over several. That way if you need to
stop one instance for any reason, you dont lose your whole cache, you only
lose part of it, which can be recreated from the database. <BR><BR>One of my
smaller projects has a finite set of data that is only about 20mb in size, it
will change frequently, but not get any larger or smaller. I actually
use two memcached instances on different machines (each with 20mb
allocated), and the keys are distributed over both (by the
client). <BR><BR>This has worked quite adequately for me for small
and large projects. <BR><BR>Replication sounds like a simple
thing, but in implementing it, there are a LOT of things that become
issues.<BR><BR>
<DIV><SPAN class=gmail_quote>On 10/24/07, <B class=gmail_sendername>Marcus
Bointon</B> <<A
href="mailto:marcus@synchromedia.co.uk">marcus@synchromedia.co.uk</A>>
wrote:</SPAN>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">On
23 Oct 2007, at 20:17, Brian Beuning wrote:<BR><BR>> One instance of
memcached could handle our tiny 400 MB with no<BR>> problem.<BR>> It
can probably even handle the load of 100 processes hitting it.<BR>> But I
am <BR>> concerned if memcached went down then we would miss our fixed
time<BR>> window.<BR>><BR>> Ideally we would like to have a few
memcached instances each with a<BR>> full<BR>> copy of the 400
MB. The Wiki says memcached does not do replication.
<BR><BR>Seems like the memcachedb project mentioned on here recently might
be<BR>a good fit for you. It's essentially a memcache front-end with a
bdb<BR>back end, so can survive restarts etc, while still serving some scary
<BR>performance numbers.<BR><BR>Marcus<BR>--<BR>Marcus
Bointon<BR>Synchromedia Limited: Creators of <A
href="http://www.smartmessages.net/">http://www.smartmessages.net/</A><BR>UK
resellers of info@hand CRM solutions<BR><A
href="mailto:marcus@synchromedia.co.uk">marcus@synchromedia.co.uk</A> | <A
href="http://www.synchromedia.co.uk/">http://www.synchromedia.co.uk/</A><BR><BR><BR></BLOCKQUOTE></DIV><BR><BR
clear=all><BR>-- <BR>"Be excellent to each other" </BLOCKQUOTE></BODY></HTML>