(NOTE: I posted this to my blog as well: <a href="http://www.feedblog.org/2006/10/idea_hierarchy_.html">http://www.feedblog.org/2006/10/idea_hierarchy_.html</a> )<br><br>This is an idea I've been kicking around for a while and wanted some feedback. Memcached does an amazing job as it is but there's always room for improvement.
<br><br>Two areas that memcached could be improved are local acceleration and large capacity support (terabyte range)<br><br>I believe this could be done through a "hierarchy of caches" with a local in-process cache used to buffer the normal memcached and a disk-based memcached backed by berkeley DB providing large capacity.
<br><br>The infrastructure would look like this:<br><br><code><br>in-process memcached -> normal memcached -> disk memcached<br></code><br><br>The in-process memcached would not be configured to access a larger memcached cluster. Clients would <b>not</b> use the network to get() objects and it would only take up a small amount of memory on the local machine. Objects would not serialize themselves before they're stored so this should act like an ultrafast LRU cache as if it were a native caching system within the VM.
<br><br>Since it's a local it should be MUCH faster than the current memcached.<br><br>Here are some benchmarks of APC-cache vs memcached.<br><br><a href="<a href="http://www.mysqlperformanceblog.com/2006/09/27/apc-or-memcached/">
http://www.mysqlperformanceblog.com/2006/09/27/apc-or-memcached/</a>"><a href="http://www.mysqlperformanceblog.com/2006/09/27/apc-or-memcached/">http://www.mysqlperformanceblog.com/2006/09/27/apc-or-memcached/</a>
</a><br><a href="<a href="http://www.mysqlperformanceblog.com/2006/08/09/cache-performance-comparison/">http://www.mysqlperformanceblog.com/2006/08/09/cache-performance-comparison/</a>"><a href="http://www.mysqlperformanceblog.com/2006/08/09/cache-performance-comparison/">
http://www.mysqlperformanceblog.com/2006/08/09/cache-performance-comparison/</a></a><br><br>Long story short a local cache can be 4-8x faster than normal memcached.<br><br>The local in-process cache would be available on every node within this cluster and act as a L1 cache for ultra fast access to a small number of objects.
<br><br>I'm not sure all languages would support this type of cache because it would require access to and storage of object pointers. I believe you can do this with Java by hacking JNI pointers directly but I'm not 100% certain.
<br><br>This cache would be configured to buffer a normal memcached cluster. We're all familiar with this type of behavior so I won't explain this any further.<br><br>The third component in the is a distributed memcached daemon which uses Berkeley DB (or another type of persistent hashtable) for storage instead of the normal slab allocator. While this might seem like blasphemy to a number of you I think it could be useful to a number of people with ultra large caching requirements (hundreds of gigs) which can't afford the required memory.
<br><br>There's already a prototype implementation of this in Tugela Cache:<br><br><a href="http://meta.wikimedia.org/wiki/Tugelacache">http://meta.wikimedia.org/wiki/Tugelacache</a><br><br>For optimal performance the memcached driver would have to do parallel and concurrent getMulti requests so that each disk in the system can seek at the same time. There are a number of memcached implementations (including the Java impl which I've contributed to) which fetch in serial. Since memcached is amazingly fast this really hasn't shown up in any of my benchmarks but this would really hinder a disk-backed memcached.
<br><br>This would provide ultra high capacity and since the disk seeks are distributed over a large number of disks you can just add spindles to the equation to get higher throughput.<br><br>This system would also NOT suffer from disk hotspots since memcached and the local in-memory memcached would buffer the disk backend.
<br><br>From a non-theoretical perspective the local cache could be skipped or replaced with a native LRU cache. These are problematic though due to memory fragmentation and garbage collection issues. I use a local LRU cache for Java but if I configure it to store too many objects I can run out of memory. It also won't be able to reache the 85% capacity we're seeing with the new memcached patches.
<br><br>I might also note that since it's now backed by a persistent disk backend one could use Memcached as a large <a href="<a href="http://labs.google.com/papers/bigtable.html">http://labs.google.com/papers/bigtable.html
</a>">distributed hashtable</a> similar to <a href="<a href="http://glinden.blogspot.com/2005/09/googles-bigtable.html">http://glinden.blogspot.com/2005/09/googles-bigtable.html</a>">Bigtable</a>.
<br><br>Thoughts? I think this idea could be very compelling. <br><br>-- <br>Founder/CEO <a href="http://Tailrank.com">Tailrank.com</a><br>Location: San Francisco, CA<br>AIM/YIM: sfburtonator<br>Skype: burtonator<br>Blog:
<a href="http://feedblog.org">feedblog.org</a><br>Cell: 415-637-8078