chunks?
Lamont Granquist
lamont at scriptkiddie.org
Wed May 30 17:55:19 UTC 2007
Does MogileFS chunk storage like GoogleFS does? One of the strengths of
GoogleFS's design is that it writes out storage in 64MB chunks on the
chunkserver. If you've got thousands of little 10kB files they'll get
coalesced into 64MB chunk files. When a fileserver needs to be replicated
the chunkfiles can be replicated which means you can stream 64MB at a
time. A back of the envelope calculation of the total time to stream 64MB
is:
4 ms + 64 MB / 30 MBs ~= 2 seconds
versus unchunked seeking every 10kB:
6553 * 4 ms + 64 MB / 30 MBs ~= 26 seconds
So how does MogileFS do this? If I fill up 2TB with 10kB blobs of data
under MogileFS how long does it take to get all that data off the disk?
Does it have to do a seek for every 10kB?
More information about the mogilefs
mailing list