dormando at rydia.net
Sat Apr 12 21:51:33 UTC 2008
> has anyone added in memcached in front of the tracker db? seems like
> another layer of caching could be thrown in to save some of the work
> (although i don't know how the workload is really...) and of course
> sharding and just using mysql proxy/multiple servers... seems like
> eventualy too many files you'll have to scale the tracker db too. but
> i suppose that is expected.
Yeah. We just haven't had to scale out the DB yet. It'd be a snap from
multiple design points though.
The trackers support internally using memcached to cache paths, but most
folks use external patch caching:
1) cache paths returned from the tracker in memcached, fetch from app
(don't have to talk to tracker at all for a cache hit)
2) and/or use perlbal's header/patch caching stuff. Internal to perlbal
so it'll skip hitting the app/memcached entirely.
> is perlbal required?
Nope. You do need something on the storage nodes to update io statistics
though. Could be anything.
> i have nginx right now running as my mogstored http server. i saw it
> can be used in its place... it was a piece of cake to setup. where
> else would perlbal be required at this point?
You need something with reverse proxy redirection (I guess nginx does
this?) in front of your whole kit. So your app/whatever returns paths to
perlbal/nginx/whatever and will serve up the actual file from mogilefs.
> i am looking into suggesting and trying to get development started (if
> it makes sense) for a mogilefs-aware plugin to nginx. something like
> the X-Sendfile header in Lighty (or X-Accel-Redirect in nginx
> already) but you can give it a mogilefs key instead... feed the file
> directly to the webserver and release PHP/etc. from it (just like
> X-Sendfile stuff does for normal files...)
Yup, same idea as the default perlbal stuff. Kinda.
More information about the mogilefs