MogileFS limits...

[mike]

On 4/12/08, Nathan Schmidt <nschmidt at gmail.com> wrote:
> Dormando,
>
> The most intense performance hits for us have come from losing storage
> nodes and the resulting rebalance operations putting a lot of
> read/write load on the tables. Worst case we've seen was almost 20
> hours of thrashing. Not really user-affecting but it did peg a lot of
> ops charts.
>
> We're far from optimal - using UTF-8 collations/charsets which seems
> to unconditionally add a great deal of overhead to index sizes. Our

maybe i'm missing something - why do we need utf-8 if none of the
filenames are going to be utf-8? that could be a decent savings right
there.

> initial bring-up was on a clean debian box so we went with MyISAM,
> which does have its problems - index corruption, O(n) repairs, ugh.
> Functional but non-optimal. Good point about the optimize table
> routine, we probably haven't done that for a while.

I wonder how CouchDB or something like that would work as a backend.
It might possibly be able to replace MogileFS completely, if I
understood a presentation correctly. They said it could store
documents of "any size" but I have no clue how efficient etc. I just
like CouchDB because it was designed from the beginning for automatic
replication, self-healing, etc.