Perlbal crashing on a regular basis
dormando
dormando at rydia.net
Wed Jan 30 19:31:28 UTC 2008
You running perlbal as root? :)
That's not always worked for me; which is why I suggested trying the
manual increase to something significantly higher than max_connections
is set to.
Shouldn't crash if max_connections is set too low either. You're on the
latest available release, right?
-Dormando
Kevin Olson wrote:
> I was under the impression that that’s what max_connections did using
> BSD::Resource.
> Lib/Perlbal.pm:
>
> if ($key eq "max_connections") {
> return $mc->err('This command is not available unless
> BSD::Resource is installed') unless $Perlbal::BSD_RESOURCE_AVAILABLE;
> return $mc->err("Expected numeric parameter") unless $val =~
> /^-?\d+$/;
> my $rv =
> BSD::Resource::setrlimit(BSD::Resource::RLIMIT_NOFILE(), $val, $val);
> unless (defined $rv && $rv) {
> if ($> == 0) {
> $mc->err("Unable to set limit.");
> } else {
> $mc->err("Need to be root to increase max connections.");
> }
> }
> return $mc->ok;
>
> Yep, only seems to crash when max_connections is set too low.
>
>
> On 1/30/08 11:21 AM, "dormando" <dormando at rydia.net> wrote:
>
> Did you increase the fd limit appropriately? (ulimit -n 64000), ensure
> your OS has a high enough maxfiles limit, etc?
>
> It doesn't actually crash at all? It only ever freezes?
>
> -Dormando
>
> Kevin Olson wrote:
> > First of all, thank you to everyone who provided some great ideas
> yesterday
> > for running multiple instances of perlbal. In the meantime while I'm
> > implementing a solution, I've run into some other issues.
> >
> > After optimizing our perlbal installation, making sure XS headers
> were on,
> > etc, we're currently handling about 40,000 requests/min only using
> about 86%
> > of the cpu with a small memory footprint.
> >
> > But now I'm having problems with perlbal mysteriously ( and relatively
> > silently ) freezing/crashing on a regular basis.
> >
> > At first I was getting the notorious "too many open files" error
> before it
> > crashed, so I up'd max_connections from 20,000 to 50,000. Now
> it'll be
> > chugging along for 15-30 minutes, and then it'll just freeze up.
> If I set
> > PERLBAL_DEBUG to 2, what I see from one min to the next is perlbal
> accepting
> > new connections and sending them to the selector, and then just a
> bunch of
> > messages coming from ClientProxy, telling me that it's closing
> backends.
> >
> > Once it's frozen, it'll either stay that way no matter what I do, or
> > sometimes once I remove the connection load, it'll somewhat
> recover after
> > 2-10 minutes.
> >
> > Thanks again,
> > Kevin
> >
> >
> > BTW, here's the main settings from my config file:
> >
> > SERVER max_connections = 50000
> > XS enable headers
> >
> > CREATE SERVICE picserv_bal
> > SET role = reverse_proxy
> > SET pool = picserv_pool
> > SET verify_backend = on
> > SET connect_ahead = 5
> > SET persist_backend = on
> > SET plugins=stats
> > ENABLE picserv_bal
> >
>
>
>
More information about the perlbal
mailing list