code restructure; svn a little unstable
andy at state51.co.uk
Wed Jun 7 17:50:17 UTC 2006
On Wed, 2006-06-07 at 00:12 +0000, Brad Fitzpatrick wrote:
> Notice that last file? Yes, replication policy is now abstracted out, so
> people can do bizarre policies that fit their site-local biz rules/etc,
> without dirtying the core.
Excellent stuff, looking interesting.
When I added our filthy policy, I went for "soft" rules - if you can't
find an ideal device for replication, complain and replicate anyway (but
always honour the deep-down hard rules about never replicating a file to
the same host). I figured replication outside the policy was better than
no replication. I suppose it's up to a policy if it makes "hard" or
...but the problem with soft rules is you end up wanting to shuffle
things around after your resources have changed. Or you could imagine a
policy "distribute this class to as many hosts as possible", where the
ideal changes the more hosts you bring up. A similar thing happens at
the moment if you let your mogile hosts fill before plugging a new one
in. All the writes go to the new host.
So I'm wondering if there's a place for a "shuffle" task like a
replicate, even if the mindev count has been met?
Also - with bunches of mogile hosts on different networks, we've ended
up with "put" and "get" policies as well as "replicate". I've got
certain lightweight files I generate from other files, I only ever want
these put to a certain network (I never replicate them). The get policy
might be more universal however - "prefer hosts on my local network".
More information about the mogilefs