mogilefs and geographic redundancy

[Andy McFarland]

On Mon, 2006-04-24 at 11:33 -0600, Jeff Lewis wrote:
> We basically have 2 datacenters that are in different locations.  Each
> is supposed to be an exact copy of the other at any given time - within
> reason.  I can see running MogileFS in each of the 2 locations, but how
> then, do I keep them in sync?

We also run MogileFS on more than one network, attached is a very simple
patch that helps us.

We wanted to make mogile smarter about three things when working with
multiple networks:-

 * Clients getting a file should get a path on their local network if
possible
 * Clients putting a file should be able to request which network it
ends up on
 * The replicator should put the destination copy on a different network
to the source copy

The first is easy. I modified our client so after it had called
mogile_paths it filtered them looking for ones on the local network.

To make the second and third happen I noticed that both
QueryWorker::cmd_create_open and Mgd::replicate both call
Mgd::find_deviceid to work out which device to use as a destination.  

I tweaked find_deviceid to accept parameters for "try and use devices on
these networks as a destination" and "try and avoid these networks as
destinations". I changed replicate to work out which networks to avoid,
then I changed create_open and the perl api to let clients request which
networks content ends up on after a MogileFS::new_file.

mogilefsd gets a new command line parameter "--network_replicate" to
turn on the new replicate behaviour.

When find_deviceid can't find an ideal device, it will complain via the
mogile console and fall back on the old "probably pick the most empty
device" behaviour. Replicating to the same network is better than not
replicating at all. cmd_create_open in particular will try and get 3
devices, even though it typically only uses the first. If you don't have
good devices on at least 3 hosts on your ideal network, you'll get
spurious warnings.

It's all very simple and there are some other things to watch out for
when mogiling across multiple networks, but this mail is nearly longer
than the patch, so I'll stop now.

Cheers,
Andy.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: mogile_network_patch_against_r237
Type: text/x-patch
Size: 6723 bytes
Desc: not available
Url : http://lists.danga.com/pipermail/mogilefs/attachments/20060426/1dc193c7/mogile_network_patch_against_r237.bin