Comparing perlbal performance with Squid as reverse proxy with url blocking

[Brad Fitzpatrick]

Thanks for the data!

Were you using epoll?  (Linux 2.6)

Note that while we've been aware of performance issues, that hasn't been
the focus.  The things we've put effort into so far are bizarre plug-ins
(like palette-modifying GIF/PNG files) and transparent internal redirects.

We've done profiling, though, and found most (37%) of our CPU is wasted in
the HTTPHeaders class, which isn't surprising.  We tried a couple
pure-perl versions of it, but none were faster than the original.  Mark
Smith then wrote it in C with some XS.  That'll be released sometime this
week or next, and I'd love it if you did the same benchmarks again then.
So far he's found it to be 600% faster when parsing headers into a header
object (which was most of that 37%) and a couple times faster on the other
operations.

We've also found that Perl's IO::Socket::INET does some unnecessary
syscalls, as does Danga::Socket in some cases, so we're fixing those up
this week as well.

I look forward to seeing more benchmarks from you as we start focusing on
performance.

- Brad


On Wed, 3 Nov 2004, Yusuf Goolamabbas wrote:

> I wrote a very simple plugin which would block URL patterns (what
> Squid could do for nimda/code-red). I temporarily modified it to send a
> 403 response for every request. Squid was configured to talk to one
> backend and had some acls which blocked uri patterns
>
> Running perlbal on FC1 with 2.6.9 and driving it via two clients via 100
> Mbit switched lan with httperf, I wanted to see what the request rate
> perlbal could handle compared to Squid 2.5 (uses poll)
>
> httperf command line was httperf --server <SERVER-IP> --uri /block.html
> --rate 400 --num-conns 1000
>
> At best I would get a rate of 180 req/sec with one client and 80 req/sec
> with 2 clients
>
> With Squid it was 400 req/sec with one client and 200 req/sec with 2
> clients
>
> It didn't make a difference whether I enabled client persistence in
> Squid or not. Numbers were the same
>
> Plugin code attached. Not sure if this is causing grief
>
> Regards, Yusuf
>