Web logging with Perlbal

[Matthew Pitts]

On Thu, 2007-10-25 at 20:13 +0100, Jonty wrote:
> Yes, I've written a plugin to do exactly this

Cool. I've been working on something myself and wanted to get some input
on my effort so far (no paste link below)...

Basically, the plugin provides a simple "backend" interface with two
supported currently - STOMP and file. The logging declarations work like

ACCESSLOG [<service>] = <backend> <backend specifier>

So, for STOMP a config option like the following will log to a local
message queue:

ACCESSLOG = stomp localhost:61613:/queue/logs

The main thing I had problems with, was figuring out the best HOOKS to
use. 'modify_response_headers' seemed the most logical, but interesting
to me was that it doesn't seem to get fired under reverse_proxy
services, only web_server ones. Any suggestions on the best hooks to use
would be great.

Performance is not impacted as much as I thought, although I've only
been testing on my laptop. With no plugin at all and doing everything
locally, perlbal gives me about 270-290 req/sec using ab. With the
plugin logging to a local message queue, it drops to the low 200's.
Logging to a file is nearly native speed for me.

The other big thing I'm pondering is whether it would be possible to do
the actual logging in a child process (to isolate blocking IO) and use a
simple pipe to pass the logs from perlbal to the child. Would something
like this work?

Check it out.

http://rafb.net/p/LThGLV33.html

Thanks for any input!

-- 
Matthew W. Pitts
Software Engineer
mpitts at a3its.com
336.202.3913 (mobile)

A3 IT Solutions, LLC
www.a3its.com