memcached "backends" (was Re: Simple questions from memcachednewbie)

[Vincent van Leeuwen]

On 2006-10-13 02:28:34 +0530, Jeetendra Mirchandani wrote:
> Hey Andy,
> Let me try to explain a scene.
> 
> Say, the cache is all I can serve out to a user. Processing data taks
> just too much time.
> In case I see a cache miss, I dont serve out anything, but
> asynchoronously bring the data into cache, so that next time the user
> comes back, he should get the results.
> 
> In such a scene, losing a memcached instance means losing data, a lot
> of it! Hence I would want to back up the data to mysql or bdb or a
> similar persistent store.
> 
> There could be more use cases. I have encountered quite a few problems
> that have this type of requirement!
> 
> -Jeetu
> 

In this case you're trying to solve 2 seperate problems with 1 application:
1) compiling data into a useable form takes a lot of time
2) caching improves database performance

For 2) it's logical to use memcached, but 1) should be solved before 
memcached.

We use 2 databases and some memcached servers for a similar situation: 1 
database server with the real data, 1 database server which only contains the 
compiled form (or in our case: chunks of HTML, ready to be used in pages) and 
finally some memcached servers so we don't have to hit the 2nd database so 
often.

You can use whatever you want for the 2nd step: a database, a bunch of flat 
files, some fancy persistent cache thingy, as long as it's persistent. And 
then the memcached servers can go up and down all they want: the overall 
system remains pretty fast. Even if all memcached servers go down our site is 
stil useable, although it's a lot slower. And as soon as memcached comes up 
again it doesn't take long to be fast again.

I personally see the academic value of multiple backends, but I don't see the
practical value. It can only make memcached more complex (harder to maintain,
more bugs) and probably also slower then it is now, so I'm against
implementing such a thing.

Memcached is a very specific tool for a very specific purpose, and should not 
be used for purposes for which it isn't meant (at least not without realising 
the implications).


Regards,

Vincent van Leeuwen
Media Design - http://www.mediadesign.nl/