code restructure; svn a little unstable

Brad Fitzpatrick brad at danga.com
Wed Jun 7 00:12:59 UTC 2006


Mogilers,

WARNING:  don't use subversion version of MogileFS for production right
now.

I've been refactoring the mogilefsd server code a ton over the past few
days.

Whereas we used to have one 3,500 line server file, it's now broken into
separate files:

   850   3393  27481 mogilefsd
  1094   4185  36398 lib/MogileFS/Worker/Query.pm
   408   1761  15224 lib/MogileFS/Worker/Replicate.pm
   128    440   4263 lib/MogileFS/Worker/Monitor.pm
    99    316   2691 lib/MogileFS/Worker/Reaper.pm
   137    519   4810 lib/MogileFS/Worker/Delete.pm
    98    288   2445 lib/MogileFS/Worker.pm
    52    158   1172 lib/MogileFS/Connection/Client.pm
    84    242   2039 lib/MogileFS/Connection/Worker.pm
   703   3193  24557 lib/MogileFS/ProcManager.pm
   144    504   5293 lib/MogileFS/Config.pm
    13     34    242 lib/MogileFS/Sys.pm
   112    378   2706 lib/MogileFS/Util.pm
    59    245   1898 lib/MogileFS/ReplicationPolicy/MultipleHosts.pm
  3981  15656 131219 total

Notice that last file?  Yes, replication policy is now abstracted out, so
people can do bizarre policies that fit their site-local biz rules/etc,
without dirtying the core.  There's no way to configure that yet, but
it'll be a 'mogadm' command on a class, and you'll be able to set
arbitrary key/value metadata on a class too, so for;

   MogileFS::ReplicationPolicy::Geographic

You could set the 'continent' and 'country' or whatever keys on the
classes, as needed by the replication policy you're using.

Also improved in the new version is speed (deleting/replication is
faster with less pointless sleeping when there's stuff to be done), and
monitoring is a ton better:  both hosts and devices separately are
detected as failed, and info is shared among all processes in realtime,
which benefits a ton of stuff.

All that said, things are still a bit unstable and I fix up bugs I
introduced while rearranging and/or rewriting everything.

Will give an update later when it's stable.

- Brad


More information about the mogilefs mailing list