Using large files with MogileFS

Jaco atoomkern at gmail.com
Tue Sep 18 10:28:25 UTC 2007


crash log: Negative length at /usr/share/perl5/Danga/Socket.pm line 1133.

Socket.pm states (line 1127-1142):
    # if this is too high, perl quits(!!).  reports on mailing lists
    # don't seem to point to a universal answer.  5MB worked for some,
    # crashed for others.  1MB works for more people.  let's go with 1MB
    # for now.  :/
    my $req_bytes = $bytes > 1048576 ? 1048576 : $bytes;

    my $res = sysread($sock, $buf, $req_bytes, 0);
    DebugLevel >= 2 && $self->debugmsg("sysread = %d; \$! = %d", $res, $!);

    if (! $res && $! != EWOULDBLOCK) {
        # catches 0=conn closed or undef=error
        DebugLevel >= 2 && $self->debugmsg("Fd \#%d read hit the end of 
the road.", $self->{fd});
        return undef;
    }

    return \$buf;

dormando schreef:
> Can you get the crashlog from mogstored?
>
> Run it in the foreground or watch syslog/etc.
>
> -Dormando
>
> Jaco wrote:
>> Hi,
>>
>> MogileFS is running fine now, but there are still some major problems 
>> when I try to upload files bigger than 1 GB. With such large files I 
>> get errors like "Send failure: Connection reset by peer" and 
>> "select/poll returned error" and the mogstored daemon is suddenly 
>> killed. Is this normal?
>>
>> If it is normal, is there a way to automatically split large files 
>> into smaller chunks during upload using mogilefsd? I know this can be 
>> done with mogtool, but I'm looking for a more transparent method 
>> within mogilefsd.
>>
>> Thanks in advance.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/mogilefs/attachments/20070918/5c4ea599/attachment.htm


More information about the mogilefs mailing list