Patch and RFC: Perlbal OOM with buffered uploads and limited disk bandwidth. Also other upload weirdness

jbj at jbj at
Tue May 15 19:49:59 UTC 2007

On Tue, May 15, 2007 at 07:31:29PM +0000, Brad Fitzpatrick wrote:
> Interesting, but how is that not handled by this existing code in
>     # deal with chunked uploads
>     if (my $cus = $self->{chunked_upload_state}) {
>         $cus->on_readable($self);
>         # if we got more than 1MB not flushed to disk,
>         # stop reading for a bit until disk catches up
>         if ($self->{read_ahead} > 1024*1024) {
>             $self->watch_read(0);
>         }
>         return;
>     }
> That seems like it should do it already?

Unless I'm missing something, that will only help if
chunked_upload_state is not undef? (in our case it's just a huge block
of data being sent by an HTML form in a POST)

> If not, could you write a failing test for this, by starting with the
> existing buffered upload test, copying it to a new test, and making it
> just do large buffered to disk with forced delays in the Perlbal
> server at the right place (faking a slow disk), forcing the OOM?

Ok - I'll have a look at this. Any likely reason why it might OOM
after just 60MB? (servers have 2GB RAM, and plenty of it is free).


More information about the perlbal mailing list