Hackathon / Multidimensional keys / Wildcard deletes

[Ian Kallen]

On 7/8/07, Paul Querna <chip at corelands.com> wrote:
>
> On 7/7/07, Steve Grimm <sgrimm at facebook.com> wrote:
> >
> > Obviously both can be supported. But it's something to be aware of:
> > sending
> > the server a bunch of commands that take even more time than the current
> > protocol's to parse will probably have a much bigger than expected
> > negative
> > impact on memcached's CPU efficiency.
>
>
> In general, I disagree -- I don't like having multiple protocol interfaces
> to one thing like memcached -- all it means is various clients will only
> implement one of X possible protocol formats, all with different feature
> sets, creating a massive mess on the client side of memcached, something
> that isn't often discussed here.
>

I think protocol flexibility allows clients with simple requirements to dumb
down for speed and smarten up for new capabilities. I'd prefer to see "basic
memcached" remain available for the applications that require  the existing
high-throughput/low-overhead usage. There will always be cases where
response times exceeding 10ms is unacceptable and nothing fancy is required
wrt to maintenance of the cache. That said, the other cases where more
complicated cache mgt semantics are required (iterating through keys, cache
entry evisceration en mass with namespaces, tags, etc) should be accounted
for. HTTP makes a lot of sense (stateless, extensible, if the shoe fits...),
there are applications where it's perfectly OK for the cache to have 50ms
response times. Particularly where it's still order/s of magnitude faster
than an RDBMS, lower throughput is an acceptable price to pay for having
sophisticated cache management. From what I reckon, greater protocol
complexity has Moore's Law (even if it is slowing down) on its side.

Anyway, it seems like there are lot of great issues brought up by these
discussions:
* different levels of complexity in cache metrics and/or mgt
* protocol flexibility to account for the complexity levels above
* storage back-end flexibility
* for persistent stores, distributed key/value pairs that aren't just caches
but *are* the durable stores of data
* client issues and making them server-issues (re-hash policies, store
redundancy, etc)

I'm thinking that these discussions are leading to the design of a data
store server toolkit , reminiscent of the NCSA server patches leading to
Apache. Perhaps it makes sense to form a memcached foundation that can take
code donations; hopefully without all of the pomp, circumstance and politics
of the ASF. I dunno, maybe all of the ASF stuff is inevitable, it's
something to think about as well. I won't be there Monday (no OSCON this
year either, too busy) but Josh will be, he'll speak for (and code better
than) me :)
-Ian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/memcached/attachments/20070708/9531a81e/attachment.htm