key expiration for many memcached clients and DB stress

[Serhat Sakarya]

Hi Victor,

One thing you can consider to mitigate effects is to add a random factor of
about +/- 10% to the expiration time.

Another option is to invalidate those keys every 1/1000 requests. You could
use a rand() function for this or use the increment/decrement facility in
memcached.

Of course it's still going to depend on the specifics of your application
whether the above approach won't mess up the consistency of your data set.

Regards,

Serhat

On 10/20/06, Victor Gumayunov <gva at reamdaysoft.com> wrote:
>
> Hello everybody,
> please advise how to solve our problem.
>
> We're using memcached in our web project.
> The site is actually a frond-end to some database.
> Since number of users is very high, we cache DB info in memcached.
> But when some memcached key expires, many users simultaneously rush to
> the DB server stressing it to the edge.
>
> Obvious solution is to notify in some way 1 web client that certain key
> is about to expire, so the client re-fetches it from DB and updates in
> memcached, while other clients still use cached value until it is
> updated by notified client.
>
> We tried several client-side approaches to implement this scenario,
> but no one gives 100% good result.
>
> So we changed server's logics for key processing.
> Our modified memcached recognizes specially formatted key names
> and does what I described above (formatted key name contains
> various parameters for such a behavior).
>
> This works almost well, but now we (of cause) run into expected
> problem that our memcached server cannot be updated when new
> memcached version is released.
>
> Can you advise how to solve our problem (key expiration for many
> memcached clients simultaneously, which causes DB stress) without
> patches and various complicated client-side tricks?
>
> Thanks in advance.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/memcached/attachments/20061020/f62ce3cd/attachment.htm