MogileFS limits...

[mike]

On 4/12/08, dormando <dormando at rydia.net> wrote:

> The trackers support internally using memcached to cache paths, but most
> folks use external patch caching:
>
> 1) cache paths returned from the tracker in memcached, fetch from app (don't
> have to talk to tracker at all for a cache hit)

That'd be even easier... the app level should go to the
perlbal/reverse proxy box, NOT directly to the mogstored server,
right? and I'm just caching the path, and the proxy is the frontend to
the mogstoreds?

> You need something with reverse proxy redirection (I guess nginx does this?)
> in front of your whole kit. So your app/whatever returns paths to
> perlbal/nginx/whatever and will serve up the actual file from mogilefs.

The reverse proxy is meant to be the layer of abstraction in front of
all the individual mogstored's, right? and the bonus of perlbal is it
will retry the backend(s) until it finds one and was designed to
return a file hopefully no matter what is down...

If that's not right I think I'm missing a key concept here. Doesn't
help I'm running all of this on the same machine at home to test :)

> > i am looking into suggesting and trying to get development started (if
> > it makes sense) for a mogilefs-aware plugin to nginx. something like
> > the X-Sendfile header  in Lighty (or X-Accel-Redirect in nginx
> > already) but you can give it a mogilefs key instead... feed the file
> > directly to the webserver and release PHP/etc. from it (just like
> > X-Sendfile stuff does for normal files...)

> Yup, same idea as the default perlbal stuff. Kinda.

Is there a C api for mogilefs? Probably quite simple, I guess I only
need a GET style request for it. I don't know what is required (is it
up to the client to first ask the tracker, get the location, then
contact the location...?)

Just looking to get the C code for MogileFS access so I can try to pay
someone (read: motivate) to get the nginx module going. I don't know C
well enough, I would try to hack it but I would have no idea if it was
even done right, and I don't know enough to debug it properly :)