How to persist data in memcached

Steven Grimm sgrimm at facebook.com
Wed Jul 25 08:31:33 UTC 2007


Hanson Lu wrote:
> Hi,
>    I want to save all data(to file system) in memcached when 
> application exit, how to do it?
>
> AFAIK, memcached has no interface to traverse all data, right?

That's right. Memcached is a cache, not a primary data store. If you 
need to persist items to a database, you should do it incrementally as 
you populate the cache. In our environment (with a few exceptions) we 
only ever populate the cache with data we've read from a database -- 
that is, everything in the cache is guaranteed to be backed by a 
persistent store because it comes from a persistent store in the first 
place. When we update an object, we update it in the database and delete 
it from the cache, so the next reader that needs it can pull it from the 
database. That last part may or may not be optimal for your application 
(it essentially means the cache is lazy-populated) but the first part is 
a good guideline: if it's not already backed by a persistent store, 
don't put it in the cache. Follow that and you'll never need to worry 
about whether the cache can be dumped to disk.

Dumping the cache to disk is dangerous in any event. Consider:

User updates his account information
Your code updates the database
Your code updates the cache
You dump the cache to disk and start working on the cache host
User updates his account information again
Your code updates the database
Your code doesn't update the cache (because it's offline at the moment)
You bring the cache back online and it restores its state
User views his account information and gets the old data from the cache

Obviously this race condition exists to some extent even without a 
persistent dump, but it's miniscule in comparison and is much easier to 
work around in the application code. On a busy site any persistent 
snapshot of the cache is pretty much guaranteed to be stale even before 
it's finished hitting the disk.

That said, if you want a persistent store with a memcached interface, 
Google "Tugela cache" and you might find that more to your liking.

-Steve


More information about the memcached mailing list