ClientHTTPFile

Derek Boonstra dboonstra at veoh.com
Fri Nov 9 18:52:41 UTC 2007


Hi, 

I was testing the client ->new_file() option of 'largefile' as described
in the change log below.

-------------------
http://code.sixapart.com/trac/mogilefs/changeset/1113/
Message:

    Add new ClientHTTPFile client back-end. This does partial PUTs to
the store
    on write (and so avoids buffering the file in RAM) and also provides
a
    seekable read/write filehandle to a file stored mogilefsd.

    Adds a new client method (read_file) using ClientHTTPFile,
implemented
    via get_paths.

    Adds a 'largefile' option to new_file, to use ClientHTTPFile instead
of
    the NewHTTPFile backend.
-------------------

I am very interesed in this because adding data to mogstored via
NewHTTPFile is instable with very large, 150mb, files while offline
testing with LWP puts directly to the mogstored has not failed on me
(yet).  I currently want to avoid chunking in the manner that mogtool
handles the situation.

In attempts of  
  my $fh = $mogclient->new_file( $key , $class , $len , { largefile => 1
} )
 
I have been unable to get an $fh.

Also, in MogileFS-Client 'make test' fails on t/20-edit.t with the
message 
   "file handle using HTTPFile"

In lib/MogileFS/ClientHTTPFile.pm -> sub TIEHANDLE, I read
> while ( $self->{path) { 
>    [ .. Do stuff until we undefine $self->{path} ... ]
> }
> return unless $self->{path}

So, it seems that ClientHTTPFile.pm is in a state in which it will never
return a tied IO object.  My current guess is that this is intentional.

I also noticed in ib/MogileFS/ClientHTTPFile.pm -> sub WRITE that it
uses LWP with a PUT method and a Content-Range.  My own testing of range
puts directly unto mogstored did not bring any success.  

2 questions come from this 

1)
Is it possible that ClientHTTPFile.pm may work ?

2)
If mogstored doesn't support range puts, would installing the
chunked-put branch of PerlBal enable the feature ?

If you've read this far, I appreciate your time.
  thanks , db



More information about the mogilefs mailing list