File duplication question

[max goldberg]

One thing mentioned in quite a few of your presentations is using more than
one mysql server as the "master" because switching from slaves takes too
long and the cluster product being too unstable to work with, not to mention
having multiple slaves slows things down exponentially. Is there anything
you can point me to that explains how you this, or is it something as simple
as duplicating all writing queries to each master?

Thanks again!

On 1/9/06, Brad Fitzpatrick <brad at danga.com> wrote:
>
> That's exactly what LiveJournal's products do, so we don't store
> duplicated contents in MogileFS.
>
>
> On Mon, 9 Jan 2006, Justin Azoff wrote:
>
> > max goldberg wrote:
> > > Hello all,
> > [ snip :-) ]
> > > Currently I use an MD5 hash for the file name and a database back end
> to
> > > try and keep track of all the files. One of the really nice things
> about
> > > this system is that on my site file assets are duplicated a lot. Some
> > > assets are duplicated up to a thousand times. Using the MD5 setup, I
> can
> > > create `symbolic links` of sorts in the database to keep from having
> to
> > > duplicate on file space (and in turn cause more I/O to my web server).
>
> > >
> > > Is this sort of thing possible with Mogile? I haven't been able to
> find
> > > much documentation on the structure of the DB, so I can't tell if this
> > > is a standard operation or more of a "hack".
> > >
> > > Can anyone provide any insight?
> >
> > I'd say that it isn't possible using only mogilefs, but you should still
> > be able to use it..  You could keep your existing database of filename
> > -> md5sums and then add files to mogilefs with the md5sum as the key.
> > This way mogilefs only knows about the unique files, and your other
> > database takes care of the filename mappings.
> >
> > If you use memcached to cache the file -> md5sum and md5sum -> mogstored
> > path mappings your app should run nice and fast.
> >
> > --
> > - Justin
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/mogilefs/attachments/20060109/da365a8a/attachment.htm