namespace bids

[Jeremy Blain]

>I would think the overhead in terms of both memory consumption and
>compute overhead for implementing namespaces would turn memcache from
>being a clean and quick solution to being something far bigger and
>bulkier.  
>
>  
>
It probably wouldn't be that bad.  I was looking through the code/docs 
and as
far as I could tell, it doesn't seem that bad.

<snip>

>
>Our scheme is simple:     namespace.key
>
>Implementing namespace flushing might not be a huge deal if you're
>willing to create an anchor key to which is prepended ALL the keys held
>in a given namespace.  Then you just get the list of keys from the
>anchor key, and walk down the list deleting the appropriate keys.  This
>is expensive from the perspective of adding/deleting keys requires
>modifications to data referenced by another key.  Also, if the list of
>keys is huge there are obvious issues.  You could always commit the key
>names for a namespace to the database during key creation, so that
>writing to the cache is slow but subsequent reads are fast.  Then
>getting the list of keys becomes a query to the database and etc.
>
>  
>
The main issue, is flushing a subset of related entries by iterating
over it is very expensive, especially when you potentially have millions
of keys. 

flush_all works by setting a time that all cache objects that are older
than are counted as expired.  No matter how many entries, flush_all
is instant.  Being able to do that with a subset of the cache would
be very nice. 

Looking at the code (but not being very proficient with C, It seems as if
this could be implemented by adding:

1. A field to the struct of the item, that contains it's namespace.
2. A hashtable of namespace => expires  (much like flush_all, only
    per namespace)
3. Some new commands that include namespaces (maybe getn, setn, ect) to 
avoid
    messing up older clients.  Using the old functions would put the 
object in a default
    namespace.

There would be a little more cpu/memory usage, but with good lookup tables,
it shouldn't amount to much.