Cache miss stampedes

[dormando]

Brad: One day I'll use the totally sweet Gearman ;) I linked to your 
post in the FAQ.

Steven: Turns out I'm not as dumb as I thought: 
http://www.socialtext.net/memcached/index.cgi?faq#how_to_prevent_clobbering_updates_stampeding_requests

... my original FAQ entry opens (clearly, I hope) by saying what you 
suggested, then wanders off into alternatives. Which was discussed 
further on the mailing list.

So! Sorry for the back and forth :( Apparantly I forgot what I had just 
written. You're right to question why alternatives even needed 
discussion, but some of us have/had fairly awkward caching primitives 
which leads to something like this.

-Dormando

Brad Fitzpatrick wrote:
> Late to this party, but I have to mention Gearman here.
> 
> On a cache miss, instead of going to the database directly, issue a
> Gearman request with a "uniq" property, then the Gearman server will
> combine all the duplicate requests and only dispatch one worker.  The
> worker than puts it in the cache before returning to the Gearman router
> (gearmand), and then gearmand multiplexes the result back to all waiting
> callers.
> 
> 
> On Wed, 25 Jul 2007, dormando wrote:
> 
>> Hey,
>>
>> So I'm up late adding more crap to the memcached FAQ, and I'm wondering
>> about a particular access pattern:
>>
>> - Key A is hit very often (many times per second).
>> - Key A goes missing.
>> - Several dozen processes all get a cache miss on A at the same time,
>> then run SQL query/whatever, and try set or adding back into memcached.
>>
>> Sometimes this can be destructive to a database, and can happen often if
>> the expire time on the data is low for some reason.
>>
>> What approaches do folks typically use to deal with this more elegantly?
>> The better suggestion I've heard is to try to 'add' the key (or a
>> separate 'lock' key) back into memcached, and only doing the query if
>> you 'win' that lock. Everyone else microsleeps and retries a few times
>> before running the query.
>>
>> Also in most of these cases you should really run a tiered cache, with
>> this type of data being stored in a local cache and in memcached.
>>
>> This really isn't a common case, but sucks hard when it happens. In the
>> back of my mind I envision a different style 'get' command, which
>> defaults to a mutex operation on miss. So you'd do the special 'get',
>> and if you get a special return code that says it's a miss but you're
>> clear to update the data (which would release the lock?). Otherwise the
>> command could optionally return immediately, or hang (for a while) until
>> the data's been updated.
>>
>> Just throwing out ideas. Thoughts?
>>
>> -Dormando
>>
>>