Some potential usages of mogilefs

[howard chen]

Brad,

Thanks for your reply...

I hope that someday you might consider putting these kind of valuable
information on the danga's homepage, they are really really useful...

Mailing list is a good place to discuss, but not for searching...


Anyway, thanks again....


Howard

On 5/21/07, Brad Fitzpatrick <brad at danga.com> wrote:
> Howard,
>
> I often wonder if you ever try anything, or just ask questions on the
> mailing list, but I'll continue to humor you with answers.
>
> Lighttpd involves:
>
>    -- getting request for /foo/img.jpg and returning it
>
> MogileFS involves
>
>    -- getting request for /foo/img.jpg,
>    -- asking mogilefsd where /foo/img.jpg is, which;
>       -- finds out that /foo/img.jpg is currently fid #123
>       -- returns that fid #123 is on device 5 and device 7,
>          returning both URLs
>    -- proxying the request for /foo/img.jpg to
>        http://moghost/dev5/0/.../123.fid
>
> Which looks like more overhead to you?
>
> Where MogileFS wins is that you spread out your storage, and things are
> constantly replicated/safe if a single machine or host dies.
>
> That said, there's a lot you can do to make the above steps fast/cached.
> If you use Perlbal as your front-end load balancer, the part of your app
> that looks up /foo/img.jpg -> FIDs(5,7) can, when returning the list of
> FID URLs to Perlbal, instruct Perlbal to remember that mapping for 'n'
> seconds, so when the next /foo/img.jpg request comes in, probably in under
> a second on popular sites, all those steps are skipped and the request
> goes right to the mogile storage node (which can be RUNNING lighttpd
> itself).
>
> So MogileFS can be fast, but it's always going to have at least that one
> more step, sometimes two, so don't expect lighttpd speeds.  We care about
> scalability and uptime around here, not raw speed on a single node with
> ideal conditions.
>
> Use MogileFS for the right reasons.
>
> - Brad
>
>
> On Mon, 21 May 2007, howard chen wrote:
>
> > Hello,
> >
> > 1. Do you think mogilefs for serving large amount of static files,
> > rarely changed, e.g. images, javascript, css etc.
> >
> > Somethings similar to Yahoo's yimg (but not yahoofs).
> >
> > Currently we are using load balancer + rsync, which suite our need if
> > the scale is small, but in the future, if you have large amount for
> > files, rsync will become the bottleneck
> >
> > 2. Are there any performance metrics available? such as max. of
> > storage volume in real world setup, number of requests/response per
> > seconds on a modern server...I know it is difficult to compare as
> > people always has different setup, but I would like to know more abt
> > the general overhead, e.g. serving 1MB JPEG file from lighty (1.5,
> > async) vs mogilefs
> >
> >
> > Thanks for any inputs...
> >
> >
> > Howard
> >
> >
>