key expiration for many memcached clients and DB stress

[Randy Wigginton]

This is what we did.  Works very well, requires a very light weight  
process on the side.  DB is happy, memcache serves everyone.   
Otherwise the rush to the DB server can be a killer.

On Oct 20, 2006, at 5:01 AM, Ben Suffolk wrote:

> Victor,
>
> I would be set it never to expire, then have a separate process  
> updating the cache with the new data every x time period (your old  
> expiry time say).
>
> This way you always have valid data in the cache, and when it time  
> to update you only get one process hitting the database not every  
> connection that suddenly finds its not got what it wants in the  
> cache (which I think is your main problem)
>
> Ben
> On 20 Oct 2006, at 12:36, Victor Gumayunov wrote:
>
>> Hello everybody,
>> please advise how to solve our problem.
>>
>> We're using memcached in our web project.
>> The site is actually a frond-end to some database.
>> Since number of users is very high, we cache DB info in memcached.
>> But when some memcached key expires, many users simultaneously  
>> rush to
>> the DB server stressing it to the edge.
>>
>> Obvious solution is to notify in some way 1 web client that  
>> certain key
>> is about to expire, so the client re-fetches it from DB and  
>> updates in
>> memcached, while other clients still use cached value until it is
>> updated by notified client.
>>
>> We tried several client-side approaches to implement this scenario,
>> but no one gives 100% good result.
>>
>> So we changed server's logics for key processing.
>> Our modified memcached recognizes specially formatted key names
>> and does what I described above (formatted key name contains
>> various parameters for such a behavior).
>>
>> This works almost well, but now we (of cause) run into expected
>> problem that our memcached server cannot be updated when new
>> memcached version is released.
>>
>> Can you advise how to solve our problem (key expiration for many
>> memcached clients simultaneously, which causes DB stress) without
>> patches and various complicated client-side tricks?
>>
>> Thanks in advance.
>>
>