How To Distinguish Between a Crashed Server and Non-Existent Memcached Key Value?

[Andreas Vierengel]

Ian Kallen wrote:
> On 6/4/06, Andreas Vierengel <avierengel at gatrixx.com> wrote:
>> that's right, but it could lead to a nice DoS-Attack especially if (for
>> example) a web-request leads to a specific key which cannot be filled
>> from the original source, because the original source produces 'undef'
>> as well.
> 
> If there is a high frequency of null result lookup cases on the data
> origin and you're concerned with the costs of those lookups, then
> implement a negative cache.  The negative cache interval should be
> sized by how tolerant your applications are of having no result when
> in fact there may be one.
> 
> Here's some babytalk code to illustrate how to typically handle it:
> 
> rv = getMemcacheVal(key);
> if (rv == null) { // memcache said "nope"
>    rv = getExpensiveVal(key); // poor performing mysql indices, etc
>    if (rv == null) { // rdbms said "nope"
>        putNegativeCache(key); // has appropriate TTL
>    } else { // rdbms said "yep"
>        putPositiveCache(key);
>    }
> }
> return rv; // may be undef
> 
>> One solution could be that you never let this happen and always generate
>> a "reasonable" value for this key, or you could hack Cache::Memcached to
>>   add a undef-state using the flags-field.
> 
> Hacking the mecached client to handle the negative case, given the
> various tolerances that applications will have for "false negatives",
> sounds like a bad idea. When I fetch null from memcache, I really
> expect that to mean there's no cache entry for it and have that be
> distinct from a negative cache. But I'd be curious to see a general
> solution if you have one.

I use a self written client in perl, which stores undef values as the 
string "UNDEF" and set some bitfield in the flag-header to true. If I 
fetch a key from memcached an my client returns undef, i question 
another method which returns if the client returned undef because of a 
general error or if the value was simply undef. So I can distinguish 
between negative hits and errors.

--Andy

> -Ian
>