Possible use for memcache?

mike mike503 at gmail.com
Sun Nov 27 23:26:30 PST 2005

On 11/27/05, Ivan Krstic <krstic at fas.harvard.edu> wrote:
> Can you actually explain what you're trying to accomplish? High
> availability via NFS server failover? Increased performance via multiple
> NFS servers serving the same physical storage? Both? Something else
> entirely?

High availability - currently NFS does not support failover without
some device-level abstraction using LVM2 (and it might not even be
that easy) - the inodes must all be the same on both machines or it's
useless (also that method seems to imply different physical storage

Obviously performance and scalability is a concern - although not
addressed yet, since I'm wondering if this is even a worthy idea of
further investigation. If/Once something is prototyped I suppose it
could be benchmarked and such (?)

> FUSE is Linux-only.

FreeBSD has a port of it now - and I believe it is production-ready
(http://fuse4bsd.creo.hu/ for instance)

> Getting all the LOOKUP/translate handle/cache handle/avoid ESTALE
> semantics right would probably be a complicated, error-prone, tricky,
> and slow mess, particularly for someone not well versed in the subject
> matter.
> NFS talks about files in terms of opaque binary filehandles which are
> server-specific; you'd need to do all sorts of extra bookkeeping and
> synchronization to add the logic you describe. On top of that, flock()
> on NFS is local prior to Linux 2.6.12, so if any of your NFS client
> machines aren't running .12+ but use flock(), you now have different
> machines getting different locks on the same file from different servers.

That is why I am talking about a middle layer. NFS client <-> this
middle layer <-> NFS server. The middle layer would be basically the
"lock manager" as it would decide which actual NFS server is serving
the content and only allows the client to hit the physical storage for
a specific file through one specific server.

> I'm still not sure I comprehend the problem you're trying to solve.
> Relevant bibliography, however:
>  http://www.citi.umich.edu/projects/cluster_nfsv4/
>  http://www.citi.umich.edu/projects/gridnfs/index.html
>  http://www.citi.umich.edu/projects/gridnfs/draft-welch-pnfs-ops-01.txt

Yeah - NFS v4 looks promising, and it looks like they have plans
eventually to add in redundancy, HA, failover, all those good things;
but it isn't even a roadmap yet - just a "to be addressed some day"

> Memcached is not a DLM, you probably really don't want to use it as one,
> and so you might want to take the discussion to a (N)FS-oriented list
> instead.

Yeah - I've posted this to FreeBSD-hackers and -fs, just to pick their
brains. There are no GFS options for FreeBSD (although I think GEOM
Gate might be promising or possible) - no replies yet as it has only
been posted for a couple hours.

I wouldn't be considering memcached as a DLM, but more as just a way
to track which servers are advertising(?) which files.

Thanks for the replies :)

More information about the memcached mailing list