Patch and RFC: Perlbal OOM with buffered uploads and limited disk bandwidth. Also other upload weirdness

[Brad Fitzpatrick]

On Tue, 15 May 2007 jbj at forbidden.co.uk wrote:

> > That seems like it should do it already?
>
> Unless I'm missing something, that will only help if
> chunked_upload_state is not undef? (in our case it's just a huge block
> of data being sent by an HTML form in a POST)

Hah... whoops, my bad.  I was reading the wrong code.  I messed up chunk
uploads with buffered uploads.  Sounded the same in my head, I guess.

> > If not, could you write a failing test for this, by starting with the
> > existing buffered upload test, copying it to a new test, and making it
> > just do large buffered to disk with forced delays in the Perlbal
> > server at the right place (faking a slow disk), forcing the OOM?
>
> Ok - I'll have a look at this. Any likely reason why it might OOM
> after just 60MB? (servers have 2GB RAM, and plenty of it is free).

Perl likes to do that with too big of sysread/syswrites.  Don't know why.

In any case, there are two simple places to fix this, but I'd really like
a test case to make me feel better about either/both of them, so let me
know if you whip one up.

- Brad