Logging
Timo Ewalds
timo at tzc.com
Tue Dec 7 00:01:59 PST 2004
Sean Chittenden wrote:
>> I run a large site that is using memcached to a small degree at the
>> moment. Really it only caches things that are used on almost every
>> page, not things that are specific to a single page or to a single
>> user. As the database becomes the bottleneck instead of the
>> webservers as it is right now, I'll need to drop SELECTs and UPDATEs.
>> Caching things that are specific to individual pages will help drop
>> the selects, but won't drop the updates. I do alot of updates of the
>> form UPDATE ... SET count = count + 1 WHERE ... for stats and other
>> logging, that could be combined into single queries over time. The
>> various hit counters, banner view counters, etc, could all be done in
>> the form of incr some memcached entry, then once a minute zero them
>> and do a single query of the form UPDATE ... SET count = count + (sum
>> of all from the memcached entry) WHERE ..., greatly reducing the
>> UPDATE query count. How hard would it be to create a command similar
>> to incr/decr that would just reset that key to 0 and return the old
>> value?
>>
>> Does anyone have a better solution to this problem, memcached related
>> or not?
>
>
> Right now it's not possible to have memcached output anything when
> data expires, but it's an addition that could be added. The best way
> would be via syslog(3), but therein lies a huge potential for a local
> DoS if lots of data gets expired. A better approach would be to set a
> "log me on expiration" bit in the new protocol, that way there is some
> selectivity for logging. Brad, would you mind if I added that? -sc
>
I was thinking just along the lines of a zero($key) type function that
returns the old value and sets the value of that key to 0 as an atomic
operation. It would be used from a script activated once every minute or
so from cron to keep the database at least relatively up to date. You'd
then use incr($key) on each page to update the results. I realized
afterwards though, that a zero() function seems a bit pointless. Why not
just keep the full version instead of an incremental version then
instead of count = count + (incremental as tracked by memcached), just
use count = (total as tracked by memcached) from the cron page.
That can't possibly be the optimimum solution though, as memcached is
meant to be a cache, not a statistics daemon. Does anyone know of a
better solution?
Timo
More information about the memcached
mailing list