ClientHTTPFile
Derek Boonstra
dboonstra at veoh.com
Fri Nov 9 18:52:41 UTC 2007
Hi,
I was testing the client ->new_file() option of 'largefile' as described
in the change log below.
-------------------
http://code.sixapart.com/trac/mogilefs/changeset/1113/
Message:
Add new ClientHTTPFile client back-end. This does partial PUTs to
the store
on write (and so avoids buffering the file in RAM) and also provides
a
seekable read/write filehandle to a file stored mogilefsd.
Adds a new client method (read_file) using ClientHTTPFile,
implemented
via get_paths.
Adds a 'largefile' option to new_file, to use ClientHTTPFile instead
of
the NewHTTPFile backend.
-------------------
I am very interesed in this because adding data to mogstored via
NewHTTPFile is instable with very large, 150mb, files while offline
testing with LWP puts directly to the mogstored has not failed on me
(yet). I currently want to avoid chunking in the manner that mogtool
handles the situation.
In attempts of
my $fh = $mogclient->new_file( $key , $class , $len , { largefile => 1
} )
I have been unable to get an $fh.
Also, in MogileFS-Client 'make test' fails on t/20-edit.t with the
message
"file handle using HTTPFile"
In lib/MogileFS/ClientHTTPFile.pm -> sub TIEHANDLE, I read
> while ( $self->{path) {
> [ .. Do stuff until we undefine $self->{path} ... ]
> }
> return unless $self->{path}
So, it seems that ClientHTTPFile.pm is in a state in which it will never
return a tied IO object. My current guess is that this is intentional.
I also noticed in ib/MogileFS/ClientHTTPFile.pm -> sub WRITE that it
uses LWP with a PUT method and a Content-Range. My own testing of range
puts directly unto mogstored did not bring any success.
2 questions come from this
1)
Is it possible that ClientHTTPFile.pm may work ?
2)
If mogstored doesn't support range puts, would installing the
chunked-put branch of PerlBal enable the feature ?
If you've read this far, I appreciate your time.
thanks , db
More information about the mogilefs
mailing list