set_multi() and delete_multi()

[Marc]

In the latest libmemcache client I¹ve implemented delete_multi.  I haven¹t
done set_multi yet because internally we currently have little demand for
it.  With the new libmemcache design it wouldn¹t be too hard to add.  In
the new API, there is only one mc_delete function.  It takes a list of keys
and an exptime for all of them.  Delete request messages are built for all
targeted servers and queued up for asynch transmission to the remote
memcache daemon.  Results are correlated with original request by order of
arrival.  That is the first result is assumed to be for the first request to
that server, &c.  Heterogeneous multi_* ops are not supported.

set_multi is slightly more difficult if one wants to do keys with
heterogeneous mixes of flags or exptimes.  delete worked well because the
typical use case is to specify only the key.  I can easily imagine wanting
to do set_multi with different flags.  For example a php or python app needs
to set an int value, a string value and a complex array.  The first two wind
up with flags == 0 and the last with flags == php or python serialized.
To support that means the set_multi will have to take a vector of
set_message structs with key name, value, flags and exptime.  Not a huge
deal but it complicates the API.  Unlike mc_delete or mc_get which basically
are always multi-operations, there will probably have to be a separate
mc_set and mc_set_multi, with the following signatures:

int mc_set(mc_handle_t mc, const char* key, size_t nkey, const char* value,
size_t nvalue, const uint16_t flags, const uint32_t exptime);

typedef struct mc_update_req_s {
    char* key;
    size_t nkey;
    char* value;
    size_t nvalue;
    uint16_t flags;
    uint32_t exptime;
} mc_update_req_t;

int mc_set_multi(mc_handle_t mc, const mc_update_req_t* reqs, const size_t
nreqs);

The same would apply for mc_add and mc_replace.  Let me know what you think.
Given that the mc_update_req_t vector only needs to exist for the scope of
the call it shouldn¹t be difficult for the extension code (ie php or python
extension) to allocate the temporary space and initialized it prior to
calling mc_set_multi, and freeing it on return.

On 3/12/07 10:33 AM, "Steven Grimm" <sgrimm at facebook.com> wrote:

> Not on the server side, no. (I can't speak for the client side; I'm not
> the one working on that here.) We rarely do large batch deletes or sets,
> so the meager CPU savings wouldn't be worth the engineering time for us.
> 
> -Steve
> 
> 
> Michael Monashev wrote:
>> > Hi Steven,
>> >
>> > Do you plan to implement set_multi() and delete_multi() in the future?
>> >
>> > SG> The server-side CPU cost of processing a two-key get request is MUCH
>> > SG> lower than processing two single-key requests because you only have to
>> > SG> run through the parser and (usually) the libevent code once. Don't
>> > SG> remember the exact ratio off the top of my head but it's pretty steep,
>> > SG> something in the neighborhood of a 10-key get request taking only 2x >>
the
>> > SG> CPU time of a 1-key request.
>> >
>> > SG> This would be less of an issue with a well-designed binary wire
>> protocol
>> > SG> that didn't require any parsing at all (though you'd still see some
>> > SG> difference due to having to wake up for input on each request), but
>> > SG> right now you can save significant amounts of CPU time on the memcached
>> > SG> server by bundling your gets up into as few requests as possible. So >>
I'm
>> > SG> glad multi-gets aren't just a series of streamed single-gets.
>> >
>> > --
>> >
>> > Michael
>> >
>> >   
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/memcached/attachments/20070312/a29bb63c/attachment.html