Lots of DB requests when cache expires...

[Olga Khenkin]

Thank you, Brad.

I like the locks idea, and I think we're going to use it. I have only
one concern: isn't it a bit heavy memcached usage? Instead of one "get"
operation we always have two: for the object itself and for the
semaphore; and much more for "set" operation. What can you tell about
this from your experience?

I'll explain you meanwhile what solution we had until now. It included a
slight change in memcached code. When somebody asked for an expired
cache object, memcached didn't return it, but set its expiration time to
a minute from "now". The first process that requested the object had
this minute to bring it from DB and save back to cache, when all other
processes received the old, slightly outdated, object.

This patch served us well with old memcached version. But the cost of
upgrading memcached is too high. Besides, the patch didn't solve the
cases of objects that do not exist in cache (new or deleted), when your
scheme solves this problem. So I think we're going to give it a try -
and if yes, I'll surely tell you how it works for us.

Thanks again,
Olga.

-----Original Message-----
From: Brad Oaks [mailto:bradoaks at gmail.com] 
Sent: Wednesday, October 10, 2007 12:59 PM
To: Olga Khenkin
Cc: memcached at lists.danga.com
Subject: Re: Lots of DB requests when cache expires...

Hi Olga,

You ask a good question.  Have a look at the solution described here
http://www.socialtext.net/memcached/index.cgi?faq#how_to_prevent_clobber
ing_updates_stampeding_requests
and let us know if that works for you.

The solution that I had come up with and still use is described at
http://lists.danga.com/pipermail/memcached/2007-July/004810.html

Best of luck,
--bradoaks

On 10/10/07, Olga Khenkin <olga at metacafe.com> wrote:
> Hi,
>
> When some popular object expires from memcached, all its customers try
to
> retrieve the object from database simultaneously, and the database
becomes
> overloaded.
>
> If this object is requested about 100 times in a minute, and if the
process
> of receiving it from DB, processing and storing in cache takes about a
> minute, then the database receives 100 identical queries.
>
> Do you have this problem? How do you handle it?
>
> Thank you in advance,
>
> Olga.