tag proposal

[dormando]

Dustin Sallings wrote:
> 
> On Oct 4, 2007, at 6:25 , Tobias Lütke wrote:
> 
>> Once tagging is available however collections can be cached directly.
>> Collections will be tagged with the key for each article contained.
>> Very complex queries can now be simply cached dumped into memcached
>> because they can be invalidated by a simple command.
>> Simply override the cached key for the article by re-setting it and
>> issue a tag invalidation on the same key:
>>
[...snip...]
> 
>     I think that's *a* use, but I don't know that it's necessarily the 
> primary use.  It's more of an argument for cleaning up tags when we're 
> done with them, though.  It doesn't change much of the original proposal 
> other than confirming that we do, indeed need to clean them up.

(Apologies if none of what I say makes sense. I'm trying).

I was expecting the number of tags to be a tenth of less than the number 
of keys, in general. This is a bad assumption.

For what Tobias is saying:

obj 'article' gets tag "article:id" (a:5, a:10, etc).

obj 'articles_month' gets all tags of articles it owns. (a:5, a:10, etc)

Then invalidation on a:5 bubbles upwards. I believe this is assuming 
'articles_month:2007_05' contains the actual _content_ of all the 
'article' objects? So you may avoid the normal indirection?

I guess this isn't always practical, but:

It should be the same amount of work to have cache relational maps 
within code. An 'article_month' object relies on 'article' objects, and 
at least in this case any 'article' object can figure out which month 
it's in. So an invalidation on an article can call down the chain of 
dependencies with itself as the argument. If the dependency can always 
figure out how it relates to this object and rebuild or expire itself, 
that completely removes the need for tags.

Having a hard time coming up with counter examples, but I'm sure this 
doesn't always work :\ So tags it is.

There's certainly a big concern about using tags and storing data 
coupled with the collections, since you're reducing the cache/ram 
efficiency.

Even if you're not, that's still a huge amount of tag space used... I 
see some memory alloc concerns, which I'll follow up with.

-Dormando