copy cmd proposal

John Berthels jjberthels at gmail.com
Fri Nov 9 14:04:22 UTC 2007


Hi folks,

I'd like to add a copy_file cmd to mogilefs (assuming there isn't one
already?).

While it's possible with a read/write loop at the moment, it would be
possible for a mogile worker to be more efficient and either:

- if src has > 1 replica, steal and DAV MOVE a replica, then force
replicate on src & dst to get back to replication policy

- if src has only one replica and device holding it permits, use DAV
COPY to create another (I/O is local to one storage server), then
force replicate

- fall back to read/write loop otherwise

this would be a potentially long-lived operation, so I thought it
could be implemented as a new 'start_copy_file' protocol cmd.


Client API could be:

MogileFS::Client::copy_file($src_domain, $src_key, $dst_domain,
$dst_key, {timeout => how_long_to_wait});

with implementation along these lines:

do_request cmd_start_copy_file
loop until timeout (fail) or get_paths (dst_domain, dst_key) returns a path

Or maybe it makes sense to expose start_file_copy directly, and let
the app do the polling loop. Or have it be an optional argument.

It may also make sense to have the cmd_start_file_copy fail if it
can't do the DAV MOVE or DAV COPY, leaving it to the client to
fallback to the read/write loop in that case.


Does anyone have a better approach than this? Is it worth my while
coding up an implementation of this for review, or is it badly flawed
in some way?

regards,

jb


More information about the mogilefs mailing list