memcached "backends" (was Re: Simple questions from memcachednewbie)

[Randy Wigginton]

And don't forget you have to put a wrapper around memcache, such that  
it checks memcache first.  If its not there, it then hits the  
database.  Or, the client could request it first from memcache, if  
its not there, load from the DB, then put it into memcache.

Of course.... you could just modify memcache and accomplish the same  
thing.

Six of one, half dozen the other.

On Oct 12, 2006, at 5:33 PM, Troy Hakala wrote:

> I see 2 solutions to your problem without requiring a change to  
> memcached:
>
> 1. Your async process could update memcached and also store the  
> data in a database for persistence.
> 2. Make it a write-through cache by writing a simple wrapper around  
> memcached's set call and use that instead of memcached's set call.
>
>
> On Oct 12, 2006, at 1:58 PM, Jeetendra Mirchandani wrote:
>
>> Hey Andy,
>> Let me try to explain a scene.
>>
>> Say, the cache is all I can serve out to a user. Processing data taks
>> just too much time.
>> In case I see a cache miss, I dont serve out anything, but
>> asynchoronously bring the data into cache, so that next time the user
>> comes back, he should get the results.
>>
>> In such a scene, losing a memcached instance means losing data, a lot
>> of it! Hence I would want to back up the data to mysql or bdb or a
>> similar persistent store.
>>
>> There could be more use cases. I have encountered quite a few  
>> problems
>> that have this type of requirement!
>>
>> -Jeetu
>>
>> On 10/12/06, Andrew Harbick <aharbick at aharbick.com> wrote:
>>> I've been watching this thread and I don't quite understand the  
>>> interest
>>> in having a different "backend" for memcache.  I'm using it as a  
>>> caching
>>> layer (as I expect most people are) between my application and  
>>> database.
>>>   In my case there is no reason to use a different "backend"  
>>> other than
>>> memory.  I'm just curious about your applications.
>>>
>>> Andy
>>>
>>> Randy Wigginton wrote:
>>> > I've customized MemCached with a (local) mysql backend.  It  
>>> works for
>>> > our purposes, but is not a general approach.
>>> >
>>> > I found that BDB was surprisingly slow.  MySQL runs in its own  
>>> thread,
>>> > which gives it a nice processing advantage.
>>> >
>>> > The idea of a "pluggable" backend is positively brilliant.  Should
>>> > probably be a part of the next release.
>>> >
>>> > On Oct 12, 2006, at 1:51 AM, Marcus Bointon wrote:
>>> >
>>> >> On 12 Oct 2006, at 06:29, Jeetendra Mirchandani wrote:
>>> >>
>>> >>> Tugela actually replaces the in-memory store with BDB
>>> >>> Ideally i would just want a transactional BDB as a backing  
>>> store, and
>>> >>> still have the usual memcached
>>> >>
>>> >> Wouldn't sharedance (http://sharedance.pureftpd.org/) achieve  
>>> much the
>>> >> same thing? Though it's not related to memcache, it  
>>> effectively does
>>> >> the same thing, but replaces the in-memory store with regular  
>>> file
>>> >> storage. The only thing I'd like on sharedance would be its  
>>> own RAM
>>> >> cache over the top of its file-based storage. The author suggests
>>> >> storing it on a RAM-based file system, but that loses you the  
>>> main
>>> >> advantages of sharedance (unlimited capacity, better  
>>> persistence) -
>>> >> you may as well be using memcache.
>>> >>
>>> >> All these things seem so similar, it's a wonder that we don't  
>>> see a
>>> >> hybrid solution with pluggable back-ends.
>>> >>
>>> >> Marcus
>>> >> --Marcus Bointon
>>> >> Synchromedia Limited: Creators of http://www.smartmessages.net/
>>> >> marcus at synchromedia.co.uk | http://www.synchromedia.co.uk/
>>> >>
>>> >>
>>> >
>>>
>>
>>
>> -- 
>> Regards,
>> Jeetu
>> http://www.cse.iitb.ac.in/~jeetu
>>
>> "Reality is merely an illusion, albeit a very persistent one."
>