Automatically maintaining cache consistency

[Mohammad J]

> It can't be done, at least not in a precise way.  There is no way to
> know exactly which data a SQL statement will affect without actually
> being able to fully execute the statement the same way your database
> would, at which point all the speed advantage is gone.

You are correct that one of the limitations of any analysis program is that
unless it does full SQL parsing, there is no way to be sure your system
isn't placed in an inconsistent state.  However, in my application, updates
are only performed using the primary key of the tables being updated (for
performance reasons).  This is true in all cases in my application, and thus
it was relatively simple to write an analysis application that identified
all queries that make updates.


The method is not meant to provide a speed benefit, but rather a
maintainability benefit.  If your application is large enough, the
maintainability penalty of using memcached becomes large.  As the site
points out, some applications (eg. Wikipedia/LiveJournal) have resorted to
keeping a text file that lists all the memcached keys.  Any time developers
are making updates, they need to consult the text file (as well as the code)
to make sure their changes aren't going to allow the system to be placed in
an inconsistent state.  In some cases, the method provides a slight (very
small) decrease in performance, but in the system I'm working on, the
penalty is very small, and only incurred in a number of specific scenarios.

It's clear that this isn't the ideal solution to the problem of maintaining
cache consistency, but I've found it quite effective in eliminating the
maintainability penalty.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/memcached/attachments/20070829/1fbf12c9/attachment-0001.html