memcached "backends" (was Re: Simple questions from memcachednewbie)

[Troy Hakala]

I see 2 solutions to your problem without requiring a change to  
memcached:

1. Your async process could update memcached and also store the data  
in a database for persistence.
2. Make it a write-through cache by writing a simple wrapper around  
memcached's set call and use that instead of memcached's set call.


On Oct 12, 2006, at 1:58 PM, Jeetendra Mirchandani wrote:

> Hey Andy,
> Let me try to explain a scene.
>
> Say, the cache is all I can serve out to a user. Processing data taks
> just too much time.
> In case I see a cache miss, I dont serve out anything, but
> asynchoronously bring the data into cache, so that next time the user
> comes back, he should get the results.
>
> In such a scene, losing a memcached instance means losing data, a lot
> of it! Hence I would want to back up the data to mysql or bdb or a
> similar persistent store.
>
> There could be more use cases. I have encountered quite a few problems
> that have this type of requirement!
>
> -Jeetu
>
> On 10/12/06, Andrew Harbick <aharbick at aharbick.com> wrote:
>> I've been watching this thread and I don't quite understand the  
>> interest
>> in having a different "backend" for memcache.  I'm using it as a  
>> caching
>> layer (as I expect most people are) between my application and  
>> database.
>>   In my case there is no reason to use a different "backend" other  
>> than
>> memory.  I'm just curious about your applications.
>>
>> Andy
>>
>> Randy Wigginton wrote:
>> > I've customized MemCached with a (local) mysql backend.  It  
>> works for
>> > our purposes, but is not a general approach.
>> >
>> > I found that BDB was surprisingly slow.  MySQL runs in its own  
>> thread,
>> > which gives it a nice processing advantage.
>> >
>> > The idea of a "pluggable" backend is positively brilliant.  Should
>> > probably be a part of the next release.
>> >
>> > On Oct 12, 2006, at 1:51 AM, Marcus Bointon wrote:
>> >
>> >> On 12 Oct 2006, at 06:29, Jeetendra Mirchandani wrote:
>> >>
>> >>> Tugela actually replaces the in-memory store with BDB
>> >>> Ideally i would just want a transactional BDB as a backing  
>> store, and
>> >>> still have the usual memcached
>> >>
>> >> Wouldn't sharedance (http://sharedance.pureftpd.org/) achieve  
>> much the
>> >> same thing? Though it's not related to memcache, it effectively  
>> does
>> >> the same thing, but replaces the in-memory store with regular file
>> >> storage. The only thing I'd like on sharedance would be its own  
>> RAM
>> >> cache over the top of its file-based storage. The author suggests
>> >> storing it on a RAM-based file system, but that loses you the main
>> >> advantages of sharedance (unlimited capacity, better  
>> persistence) -
>> >> you may as well be using memcache.
>> >>
>> >> All these things seem so similar, it's a wonder that we don't  
>> see a
>> >> hybrid solution with pluggable back-ends.
>> >>
>> >> Marcus
>> >> --Marcus Bointon
>> >> Synchromedia Limited: Creators of http://www.smartmessages.net/
>> >> marcus at synchromedia.co.uk | http://www.synchromedia.co.uk/
>> >>
>> >>
>> >
>>
>
>
> -- 
> Regards,
> Jeetu
> http://www.cse.iitb.ac.in/~jeetu
>
> "Reality is merely an illusion, albeit a very persistent one."