Patch: Perlbal HTTP 1.1 Chunked Backend Support (Was: Anyterm
daemon won't work under perlbal)
Jonty
jonty at last.fm
Thu Mar 13 15:16:26 UTC 2008
Whoops, I accidentally generated that patch against 1.59. It should only
have been offset differences though.
I've regenerated the patch against perlbal trunk (rev 769), you can find
it here:
http://playground.audioscrobbler.com/jonty/perlbal_chunked_backend_support.diff
Apologies for that,
Jonty
On Thu, 2008-03-13 at 10:48 -0400, Daniel Risacher wrote:
> What version of perlbal is this for? It didn't apply clean on 1.60 or 1.70.
>
> On 3/13/08, Jonty <jonty at last.fm> wrote:
> > Hi All,
> >
> > I found a fairly horrible memory leak in the last version of the patch,
> > you can find the new version here:
> >
> > http://playground.audioscrobbler.com/jonty/perlbal_chunked_backend_support.diff
> >
> > I suspect this memory leak also exists for chunked uploads, but I've not
> > had time to test yet.
> >
> > Tests and doc for the above patch will be posted soon. I'd appreciate
> > people testing it in other setups if possible - we're running it on our
> > live cluster, but variety breeds bugs...
> >
> > Cheers,
> > Jonty
> >
> >
> > On Mon, 2008-03-10 at 19:12 +0000, Jonty wrote:
> > > Evening,
> > >
> > > Sorry for the horrendous delay, I've only just found time to fully test
> > > the rewritten version in production.
> > >
> > > Anyway, this introduces two new per-service tunables:
> > >
> > > SET backend_http_version = (1.0|1.1)
> > >
> > > Specifies the HTTP version to be sent to the backend, as chunked
> > > encoding requires HTTP 1.1 and perlbal used 1.0 previously.
> > > Defaults to HTTP 1.0 for backward compatibility with old backends
> > > communicating with perlbal.
> > > Currently takes "1.1" or "1.0" as a value. Must be set to "1.1" to
> > > enable chunked requests to backends.
> > >
> > > SET buffer_chunked_backend_response = (false|true)
> > >
> > > Turns on buffering of the entire HTTP response from the backend for
> > > chunked responses in order to set the Content-Length header for the
> > > de-chunked response.
> > > Default false, as this can require a significant amount of memory for
> > > large responses and delays delivery of the response to the client.
> > >
> > > Currently this does not support proxying chunked responses to chunked
> > > clients, however I may add this ability in the near future.
> > >
> > > This also introduces a hook when backends are closed, making it simple
> > > to log the reason for the closure - this is invaluable when debugging
> > > connection problems to backends.
> > >
> > > Hope this meets with approval.
> > >
> > > Cheers,
> > > Jonty
> > >
> > >
> > > On Sat, 2008-03-08 at 19:59 -0800, Brad Fitzpatrick wrote:
> > > > On Tue, Feb 5, 2008 at 4:28 PM, Jonty <jonty.wareing at gmail.com> wrote:
> > > > On Feb 5, 2008 5:53 PM, Daniel Risacher <magnus at alum.mit.edu>
> > > > wrote:
> > > > > Anyone feel like implementing chunked encoding on the
> > > > backend?
> > > >
> > > >
> > > > I implemented chunked encoding to backends about six months
> > > > ago, and
> > > > we've been using it in production solidly since.
> > > >
> > > > I've been meaning to clean up the patches and submit them back
> > > > upstream (along with many, many others) but have had no time -
> > > > I'm not
> > > > entirely happy with them yet (there's a fair bit of code
> > > > duplication
> > > > with the chunkeduploadstate stuff).
> > > >
> > > > However I should have time to tidy up those patches this week
> > > > if all
> > > > goes well, so keep your eyes open.
> > > >
> > > > Yeah, I'd also be interested!
> > > >
> > > > - Brad
> > > >
> > > >
> >
More information about the perlbal
mailing list