Allocating MySQL cluster nodes for MogileFS database

[dormando]

The disk storage for Cluster in 5.1 isn't so fast or even that great 
(admitted by MySQL).

You still need fast disks to deal with MySQL Cluster, it is an "in 
memory" hash table type database, but the dataset is backed to disk, and 
you need to journal all writes to the redo logs. If you can't write to 
the redo logs faster than the writes come in, MySQL Cluster will 
unceremoniously corrupt itself.

I have no idea if that last bit's been fixed in 5.1, but it's pretty 
serious ;)

As Artur said, you're much better off going to MySQL Cluster once 
needed. GaiaOnline's using just one master/slave replicated cluster for 
mogilefs and it's managing 20+ million files for us so far. 8G of RAM, 
harddrives galore, etc.

IMHO we'll be better off with MogileFS handling clustering of 
file/file_on anyway.

-Dormando

Jared Klett wrote:
> hi Jonathan,
> 
> 	I had the same thought, initially, but as it turns out in the
> latest/greatest (gulp) version of MySQL cluster it's possible to use
> disk storage on your data nodes:
> 
> http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-disk-data.html
> 
> 	so in this case, SATA RAID-10 with perhaps 10K rpm drives
> wouldn't be a bad idea, assuming your dataset is too large to fit in
> memory.
> 
> cheers,
> 
> - Jared 
> 
> -----Original Message-----
> From: Jonathan Steinert [mailto:hachi at kuiki.net] 
> Sent: Wednesday, February 07, 2007 2:17 PM
> To: Arthur Bebak
> Cc: Jared Klett; mogilefs
> Subject: Re: Allocating MySQL cluster nodes for MogileFS database
> 
> Arthur Bebak wrote:
>> Jared Klett wrote:
>>
>>>     and the MySQL cluster storage nodes:
>>>
>>> * Dual Opteron 2.4 Ghz (Troy)
>>> * 8 GB memory (may upgrade to 16 before going into production)
>>> * 3ware 8006-2LP (RAID-1 for system)
>>>
>>>     am I way off base here? asking for trouble (performance 
>>> problems)? if so, what should I move off the file servers to other 
>>> machines?
>> I suspect your file servers are fine, though given that MogileFS 
>> copies its files into memory (e.g. the get_file_data call) the more 
>> memory the better.
>>
>> The only other comment I would make is to configure your MySQL boxes 
>> with Raid 10.
>> You want high IO rates, particularly if your DB size will be large, 
>> and of course you want the reliability.
>>
> I see a lot of talk about using ndb (Mysql Cluster) here, and while I'm
> not exactly sure of the state of this in MySQL 5.1,
> http://dev.mysql.com/doc/refman/5.1/en/mysql-cluster-basics.html still
> says that this is an in-memory storage system. Assuming that is the
> case, then RAID 10 vs RAID 1 on the MySQL data storage nodes will make
> no difference at all.
> 
> I could be all wet on this subject, and someone should correct me if I
> am, but this is what I see in your setup.
> 
> --Jonathan