seeking for opinions on multi-storage support

[Roy Lyseng]

Hi Toru,

I would like to see the wrapper functions being removed, as this will 
reduce the maximum stack depth. They are also mostly useless now without 
the single-thread implementation...

An efficient synchronization mechanism is a key success factor for an 
engine. To me, this means that the engine must take care of the mutexing.

If the do_ functions are called directly, this means that memcached is 
calling a *different* function than the mt_ function, and hence these 
should be different (loadable) engine functions. Alternatively, control 
over synchronization may be governed by an argument.

Are you also going to allow multiple engines to be loaded simultaneously?

Thanks,
Roy

Toru Maesaka wrote:
> Hey guys,
> 
> Lately I've been doing a fair amount of thinking and planning on how
> to do a clean job of supporting multi storage engines for memcached.
> 
> As a first step I've eliminated the macros that acted as an alias for
> whether to use the mutex wrappers (mt_*) or the actual implementation
> (do_*), by making memcached compile only in multi-thread mode. This
> code is currently in Dustin's binary branch.
> 
> Now, what this means is that we can now make either the wrappers or
> the implementation functions loadable (or both but this isn't really neat
> imho).
> 
> At the hackathon we came to an agreement that the cache/storage
> related mutex should live in the storage engine. My understanding is
> that to do so, we would make the mutex wrappers loadable but the
> problem here is that we can't _just_ have wrapper functions, since
> memcached directly calls the "do_*" functions at various places,
> most likely to avoid deadlocks.
> 
> On the other hand, if we were to make the implementation functions
> loadable, things are easy since we would nudge those functions inside
> the wrappers and the core server will operate on the engine through them.
> However this limits many possibilities since the concurrency control is
> determined by the core server.
> 
> So yeah, I personally would like to bring the mutex out to the storage
> engine but this involves a minor problem as stated above.
> 
> Does anyone have any advice/opinions on this?
> 
> Cheers,
> Toru