e at osterman.com
Fri May 11 08:45:28 UTC 2007
We implemented something quick yesterday using the fuse examples and
libmemcache (replicating that python example in C). We configured it to
just wrap an existing GlusterFS volume. Our initial test very crude test
was just to accelerate stating. Stating 18500 files over GlusterFS took
consistently ~22.5 seconds without the memcache layer. Stating the same
files (after hydrating the cache) took ~4.5 seconds consistently with
the memcache caching layer. We didn't really do anything for caching
content, no more time to play with it. . . didn't even try to do any
optimizations. I was just happy it helped validate idea.
If implemented truely as a caching solution, you shouldn't have to worry
about losing files either. Cache misses would fall back to the
filesystem and just rehydrate the cache. Or like you said, you could
have it fall back to a database instead of the filesystem. What I like
about using the filesystem as opposed to a database, is that it can be
used more generally to accelerate any exported filesystem.
Peter van Dijk wrote:
> On 9-mei-2007, at 20:21, Erik Osterman wrote:
>> Python proof of concept code.
>> It doesn't appear to cache contents, only stat.
>> Also, on a philisophical side note. I am floored by how often almost
>> identical ideas are conceived of by different people at the same
>> time. Look at the date on that blog entry!
> You have no idea how much this is scaring me right now ;)
> Over the last two weeks i spent a lot of time discussing a memcachefs
> (fuse-based) with two fellow geeks - applications that came to mind
> were (a) the smarty cache (b) php sessions; for both cases, losing
> files (as a whole, not random parts inside) is ok and readdir is
> irrelevant, which allows cutting a lot of corners.
> We're also pondering using mysql as the write-through bit, instead of
> an actual filesystem, which wouldmake the storage stable and allow
> readdir. No code so far, though.
> Cheers, Peter
More information about the memcached