The number of files MogileFS can support

[Mark Smith]

>     My application need to store monstrous amount of files(by billions) and
> the underlying system is Linux kernel 2.6+ and Ext3.
>
>     Size of those files are generally bellow 1K.
>
>     My question is, does MogileFS fit into this scenario?  If so, can anyone
> suggest a proper architecture, like number of trackers, file system instead
> of Ext3, etc.

Sure.  The limits you are going to run into will be on the database
first, as each file is indexed there.  The storage nodes themselves
won't have any problems with it if you use an appropriate filesystem.
MogileFS does separate files into hashed directories, so that
shouldn't be any problem.

Anyway, I can't see this being a particularly perfect fit for MogileFS
though.  I can't imagine it not working, but really small files like
that aren't really the use case it's been built for.  I'd love to hear
of any problems or issues or success you have, though.  If there's
anything overtly broken, I'm sure we can fix it.


-- 
Mark Smith / xb95
smitty at gmail.com