code restructure; svn a little unstable

[Andy McFarland]

On Wed, 2006-06-07 at 00:12 +0000, Brad Fitzpatrick wrote:
> Mogilers,

Mogilerator!

> Notice that last file?  Yes, replication policy is now abstracted out, so
> people can do bizarre policies that fit their site-local biz rules/etc,
> without dirtying the core.

Excellent stuff, looking interesting.

When I added our filthy policy, I went for "soft" rules - if you can't
find an ideal device for replication, complain and replicate anyway (but
always honour the deep-down hard rules about never replicating a file to
the same host). I figured replication outside the policy was better than
no replication. I suppose it's up to a policy if it makes "hard" or
"soft" rules.

...but the problem with soft rules is you end up wanting to shuffle
things around after your resources have changed. Or you could imagine a
policy "distribute this class to as many hosts as possible", where the
ideal changes the more hosts you bring up. A similar thing happens at
the moment if you let your mogile hosts fill before plugging a new one
in. All the writes go to the new host.

So I'm wondering if there's a place for a "shuffle" task like a
replicate, even if the mindev count has been met? 

Also - with bunches of mogile hosts on different networks, we've ended
up with "put" and "get" policies as well as "replicate". I've got
certain lightweight files I generate from other files, I only ever want
these put to a certain network (I never replicate them). The get policy
might be more universal however - "prefer hosts on my local network".