Pipelining weirdness?

[Jeremy James]

We've hit a recent problem with a perlbal instance sitting in front of
apache - in this case it's providing access to a local Debian
repository. apt-get attempts to use pipelining to fetch package files by
default and, despite things not obviously changing, it now fails to
fetch files (trace below (1) - approx line 618 in ClientProxy.pm).

All in all, we're not sure what's going on. We're pretty certain that
we've been moving a number of packages from this server in the last few
months, and the client versions of apt haven't suddenly changed. And
yet, it's obvious from the perlbal code that this shouldn't have ever
worked...

We've patched ClientProxy (2) to support these types of pipelining GETs.
However, the last test in 32-pipelining.t breaks - it explicitly expects
a 400 error to be returned (where the server now happily serves the two
requests in a row). Is there any reason this shouldn't be allowed? (it
probably doesn't work with POSTs, though, so could only permit this
behaviour for GETs and HEADs).


On an ironic note, I discovered on Monday that I need a selector for a
production machine that can switch on both hosts and URLs - and forgot
beforehand that you couldn't stack selectors. I'll work very soon on
extending vhost to support this. To appease the powers that be, I intend
to allow both regexp and apache-syntax, probably with different commands
(eg. URL and URLMATCH - although depending on argument parsing, it might
be easier to provide a prefix for the actual URL to indicate which
matching syntax to use).


I also intend to allow the listen command line to specify multiple IP
addresses - having to define selector settings multiple times for each
address is a real pain. I intend to do this with multiple sockets inside
TCPListener. Is this likely to cause problems?


Jeremy


=== 1) Web Traffic from apt (package and server names changed to protect
the innocent) ===

 GET /pool/main/i/imlib2/libimlib2_1.3.0.0debian1-0.1sarge1_i386.deb
HTTP/1.1
Host: our-package-server.something
Connection: keep-alive
User-Agent: Debian APT-HTTP/1.3

GET /pool/main/f/ffmpegcvs/ffmpeg_20070221-0.0sarge1_i386.deb HTTP/1.1
Host: our-package-server.something
Connection: keep-alive
User-Agent: Debian APT-HTTP/1.3

... (some more requests) ...

HTTP/1.0 400 Bad request
Content-Type: text/html
Content-Length: 53
Server: Perlbal
Connection: close

<h1>400 - Bad request</h1>
Can't pipeline to HTTP/1.0


=== 2) ClientProxy Patch ===

--- ClientProxy.pm      2007-04-24 16:21:43.000000000 +0100
+++ ClientProxy.pm.new  2007-04-24 17:15:48.000000000 +0100
@@ -768,6 +768,7 @@
     } else {
         # get the backend request process moving, since we aren't buffering
         $self->{is_buffering} = 0;
+        $self->watch_read(0);

         # if reproxy-caching is enabled, we can often bypass needing to
allocate a BackendHTTP connection:
         return if $svc->{reproxy_cache} &&
$self->satisfy_request_from_cache;

=== End ===