Patch: Perlbal HTTP 1.1 Chunked Backend Support (Was: Anyterm daemon won't work under perlbal)

Daniel Risacher magnus at alum.mit.edu
Thu Mar 13 14:48:32 UTC 2008


What version of perlbal is this for?  It didn't apply clean on 1.60 or 1.70.

On 3/13/08, Jonty <jonty at last.fm> wrote:
> Hi All,
>
>  I found a fairly horrible memory leak in the last version of the patch,
>  you can find the new version here:
>
>  http://playground.audioscrobbler.com/jonty/perlbal_chunked_backend_support.diff
>
>  I suspect this memory leak also exists for chunked uploads, but I've not
>  had time to test yet.
>
>  Tests and doc for the above patch will be posted soon. I'd appreciate
>  people testing it in other setups if possible - we're running it on our
>  live cluster, but variety breeds bugs...
>
>  Cheers,
>  Jonty
>
>
>  On Mon, 2008-03-10 at 19:12 +0000, Jonty wrote:
>  > Evening,
>  >
>  > Sorry for the horrendous delay, I've only just found time to fully test
>  > the rewritten version in production.
>  >
>  > Anyway, this introduces two new per-service tunables:
>  >
>  > SET backend_http_version = (1.0|1.1)
>  >
>  > Specifies the HTTP version to be sent to the backend, as chunked
>  > encoding requires HTTP 1.1 and perlbal used 1.0 previously.
>  > Defaults to HTTP 1.0 for backward compatibility with old backends
>  > communicating with perlbal.
>  > Currently takes "1.1" or "1.0" as a value. Must be set to "1.1" to
>  > enable chunked requests to backends.
>  >
>  > SET buffer_chunked_backend_response = (false|true)
>  >
>  > Turns on buffering of the entire HTTP response from the backend for
>  > chunked responses in order to set the Content-Length header for the
>  > de-chunked response.
>  > Default false, as this can require a significant amount of memory for
>  > large responses and delays delivery of the response to the client.
>  >
>  > Currently this does not support proxying chunked responses to chunked
>  > clients, however I may add this ability in the near future.
>  >
>  > This also introduces a hook when backends are closed, making it simple
>  > to log the reason for the closure - this is invaluable when debugging
>  > connection problems to backends.
>  >
>  > Hope this meets with approval.
>  >
>  > Cheers,
>  > Jonty
>  >
>  >
>  > On Sat, 2008-03-08 at 19:59 -0800, Brad Fitzpatrick wrote:
>  > > On Tue, Feb 5, 2008 at 4:28 PM, Jonty <jonty.wareing at gmail.com> wrote:
>  > >         On Feb 5, 2008 5:53 PM, Daniel Risacher <magnus at alum.mit.edu>
>  > >         wrote:
>  > >         > Anyone feel like implementing chunked encoding on the
>  > >         backend?
>  > >
>  > >
>  > >         I implemented chunked encoding to backends about six months
>  > >         ago, and
>  > >         we've been using it in production solidly since.
>  > >
>  > >         I've been meaning to clean up the patches and submit them back
>  > >         upstream (along with many, many others) but have had no time -
>  > >         I'm not
>  > >         entirely happy with them yet (there's a fair bit of code
>  > >         duplication
>  > >         with the chunkeduploadstate stuff).
>  > >
>  > >         However I should have time to tidy up those patches this week
>  > >         if all
>  > >         goes well, so keep your eyes open.
>  > >
>  > > Yeah, I'd also be interested!
>  > >
>  > > - Brad
>  > >
>  > >
>


More information about the perlbal mailing list