Patch: Perlbal HTTP 1.1 Chunked Backend Support (Was: Anyterm daemon won't work under perlbal)

[Jonty]

Whoops, I accidentally generated that patch against 1.59. It should only
have been offset differences though.

I've regenerated the patch against perlbal trunk (rev 769), you can find
it here:

http://playground.audioscrobbler.com/jonty/perlbal_chunked_backend_support.diff

Apologies for that,
Jonty


On Thu, 2008-03-13 at 10:48 -0400, Daniel Risacher wrote:
> What version of perlbal is this for?  It didn't apply clean on 1.60 or 1.70.
> 
> On 3/13/08, Jonty <jonty at last.fm> wrote:
> > Hi All,
> >
> >  I found a fairly horrible memory leak in the last version of the patch,
> >  you can find the new version here:
> >
> >  http://playground.audioscrobbler.com/jonty/perlbal_chunked_backend_support.diff
> >
> >  I suspect this memory leak also exists for chunked uploads, but I've not
> >  had time to test yet.
> >
> >  Tests and doc for the above patch will be posted soon. I'd appreciate
> >  people testing it in other setups if possible - we're running it on our
> >  live cluster, but variety breeds bugs...
> >
> >  Cheers,
> >  Jonty
> >
> >
> >  On Mon, 2008-03-10 at 19:12 +0000, Jonty wrote:
> >  > Evening,
> >  >
> >  > Sorry for the horrendous delay, I've only just found time to fully test
> >  > the rewritten version in production.
> >  >
> >  > Anyway, this introduces two new per-service tunables:
> >  >
> >  > SET backend_http_version = (1.0|1.1)
> >  >
> >  > Specifies the HTTP version to be sent to the backend, as chunked
> >  > encoding requires HTTP 1.1 and perlbal used 1.0 previously.
> >  > Defaults to HTTP 1.0 for backward compatibility with old backends
> >  > communicating with perlbal.
> >  > Currently takes "1.1" or "1.0" as a value. Must be set to "1.1" to
> >  > enable chunked requests to backends.
> >  >
> >  > SET buffer_chunked_backend_response = (false|true)
> >  >
> >  > Turns on buffering of the entire HTTP response from the backend for
> >  > chunked responses in order to set the Content-Length header for the
> >  > de-chunked response.
> >  > Default false, as this can require a significant amount of memory for
> >  > large responses and delays delivery of the response to the client.
> >  >
> >  > Currently this does not support proxying chunked responses to chunked
> >  > clients, however I may add this ability in the near future.
> >  >
> >  > This also introduces a hook when backends are closed, making it simple
> >  > to log the reason for the closure - this is invaluable when debugging
> >  > connection problems to backends.
> >  >
> >  > Hope this meets with approval.
> >  >
> >  > Cheers,
> >  > Jonty
> >  >
> >  >
> >  > On Sat, 2008-03-08 at 19:59 -0800, Brad Fitzpatrick wrote:
> >  > > On Tue, Feb 5, 2008 at 4:28 PM, Jonty <jonty.wareing at gmail.com> wrote:
> >  > >         On Feb 5, 2008 5:53 PM, Daniel Risacher <magnus at alum.mit.edu>
> >  > >         wrote:
> >  > >         > Anyone feel like implementing chunked encoding on the
> >  > >         backend?
> >  > >
> >  > >
> >  > >         I implemented chunked encoding to backends about six months
> >  > >         ago, and
> >  > >         we've been using it in production solidly since.
> >  > >
> >  > >         I've been meaning to clean up the patches and submit them back
> >  > >         upstream (along with many, many others) but have had no time -
> >  > >         I'm not
> >  > >         entirely happy with them yet (there's a fair bit of code
> >  > >         duplication
> >  > >         with the chunkeduploadstate stuff).
> >  > >
> >  > >         However I should have time to tidy up those patches this week
> >  > >         if all
> >  > >         goes well, so keep your eyes open.
> >  > >
> >  > > Yeah, I'd also be interested!
> >  > >
> >  > > - Brad
> >  > >
> >  > >
> >