node health

[dormando]

> I don't know of a way to do this through normal configuration or 
> plugins, but you could check out BackendHTTP.pm, the functions 
> event_write and event_read_waiting_options are the ones that currently 
> deal with the OPTIONS check.  You could modify it hit a specific URI 
> with a GET, instead... and then check for some magical "We're all good" 
> in the headers, I guess.  Keep in mind that the OPTIONS check is 
> currently when you first get a request (afaik), and it's assumed to be 
> very fast.  If your DB check takes a while it could block the first 
> unlucky user's request... if I'm reading it correctly.  Just sort of 
> stabbing here, I'm not a Perlbal pro.  I'm sure you're better off with 
> the next option.

OPTIONS is only ran when perlbal first connects to an individual apache 
process, so it's not used for the client side service. For each 
keepalive request it'll just blind forward. It's better to have a health 
check service for the (often rarer) case of the code on an individual 
machine exploding.

> Sure, your app (daemon or cron, whatever) could check your server for 
> connectivity then either ADD/REMOVE servers from a POOL via the telnet 
> management port, or change a nodelist file (which Perlbal will 
> eventually re-read).

Yup :) It'll reload the nodelist pretty fast, even! As a caveat (this at 
least _used_ to be true), once you modify a pool with ADD/REMOVE, it 
won't check the nodelist anymore.

-Dormando