Next Major Memcached release / roadmap?

[Nathan Schmidt]

Paul -


The ring buffers patch is too rough to merge as is -- we've bumped it  
down our internal priority list for the time being but were planning  
to return to it in a month or two.

Having sat on the patch for a bit, I think it will be smarter to make  
it use a prefixes (such as proposed) instead of new top-level  
commands as used in my initial patch. Best to not require a patched  
client library if possible, and I've convinced myself the prefix  
approach will work quite well once built.

In terms of extensibility, how about adding a query command to stats,  
so app code can verify server capability if necessary?

stats extensions\r\n
EXTENSIONS ringbuffer,append,changetimeout,randomlydeletemydata\r\n

For preserving performance, of course you don't want to strncmp 
('ringbuffer:') for every get. We could add standard key prefix  
signal, so if (key[0] == ':') then further checking is performed, but  
unless your keys already start with ':' you don't take much of a  
performance hit with other processes. That makes a full append look  
like this:

set :append:existing_key 0 0 8\r\n
moredata\r\n
STORED\r\n



For ring buffers, similar process. On a client lib, this would look like
$m->set(":ringbuffer:actual_key","xxxx") -- the xxxx data is a dummy  
value of strlen n where n = number of slots to allocate for the ring  
buffer. This looks a little gimpy, there's probably a more elegant  
approach.

set :ringbuffer:actual_key 0 0 4\r\n
xxxx
STORED\r\n

set :ringpush:actual_key 0 0 7\r\n
mydata1\r\n
STORED\r\n

set :ringpush:actual_key 0 0 7\r\n
mydata2\r\n
STORED\r\n

get :ringbuffer:actual_key\r\n
mydata1mydata2\r\n
END\r\n


Thanks,
-nathan


On May 30, 2007, at 4:47 AM, Paul Lindner wrote:

> I've been thinking of where memcached is now and where it might go in
> the future.  I was hoping the community could come together and define
> a roadmap of possible features to be added/incorporated in future
> memcached releases.
>
> Some patches that are floating around and are not yet committed
> include:
>
>  * non-expiring entries patch from Paul G
>  * Ring buffers patch from Nathan.
>  * Append data to existing entry from Filipe
>
> Also I have tasked myself with integrating some of Hi5's custom server
> code into memcached.  This code would include variants of the
> memcached storage architecture and some possible background worker
> threads.
>
> Open issues that need consensus:
>
>   * Protocol changes -- do we want to extend memcached protocol to
>     support new features?  If so, what is the logical way to do this.
>
>   * "Magic" values?  Do we want to allow some kind of mapping of
>     client data to variations in server behavior?
>
>     For example the non-expiration patch allows you to configure a
>     value that results in a non-expiring item.
>
>     Others have proposed that special features be enabled for a common
>     prefix -- say all keys that have a  "ringbuffer:" prefix use that
>     code path / storage engine..
>
>   * How do we accomodate new features without affecting performance.
>
>   * More? I'm sure there is..
>
>
> Hopefully this gets things going.  I hope we can agree on the right
> approach so we can incorporate as many community contributions as is
> possible and feasible.
>
>
> -- 
> Paul Lindner        ||||| | | | |  |  |  |   |   |
> lindner at inuus.com