Bizarre perlbal problem

Brad Fitzpatrick brad at danga.com
Fri Nov 18 10:35:51 PST 2005


I hear your concern, but I'm not worried about Perlbal mixing up requests
and responses.

However, what I'm asking about is different:  not that each request can go
to different servers, but that PHP isn't caching any session info data on
the CONNECTION object, not the REQUEST object.

So you got a TCP connection with a bunch of HTTP requests inside it:

   <tcp>
      <http_req />
      <http_req />
      <http_req />
      <http_req />
      <http_req />
   </tcp>

If the PHP is making the assumption that each http request is from the
same user (which is usually the case!) and caching on the TCP connection
object, then Perlbal could be confusing it, because Perlbal's persistent
connections to PHP aren't aligned with users at all.



On Fri, 18 Nov 2005, Dormando wrote:

> I don't think so...
>
> Our backend is just direct connections to PHP running behind either
> apache1 or apache2. As long as the remote user's independent requests go
> down and come back from the same server it should work. IE; If user A
> sends a request down, it just needs to get the reply to that request
> back. If user A sends a request down, but gets the response headers from
> user B, we'd get what we're seeing now.
>
> There's no reliance on the same user hitting the same exact backend
> connection for multiple requests. Before we put the LB in there was no
> keepalive at all, and users would hit one of 80 servers at random.
>
> -Alan
>
> Brad Fitzpatrick wrote:
> > Does your backend's session management assuming that for a given
> > connection, the same user will always be on that connection?  What backend
> > do you use, btw?
> >
> > If so, that's your problem, since Perlbal mixes up requests and
> > connections.  (on purpose... it's a huge optimization)
> >
> > - Brad
> >
> >
> > On Fri, 18 Nov 2005, Dormando wrote:
> >
> >
> >>(sorry, couldn't resist :P)
> >>
> >>We got HTTP 1.0 keepalives working (for the most part) late in the day
> >>yesterday. Overnight we witnessed a pretty bad glitch where our users
> >>would randomly get other user's site cookies and become logged in as
> >>someone else.
> >>
> >>It happened in a small percentage of users, but turning off the backend
> >>keepalives seems to have removed the issue. We're still investigating on
> >>our end, but I'm having a hard time even speculating how that happened;
> >>was perlbal sending back responses from clients other than the
> >>requestor? Any insight?
> >>
> >>Other things that came to mind:
> >>  - It's possible sometimes we return an improper content-length.
> >>  - If a client connection closes before reading any data back from its
> >>connection, does perlbal always junk the backend request before reusing it?
> >>
> >>I do have a weird routing setup in order to have two perlbal processes
> >>running on the same IP address, but since that's just on the frontend
> >>and only for incoming requests, I can't see how the responses would get
> >>switched... If they were, it'd happen a lot more often than with what we
> >>saw.
> >>
> >>Thanks,
> >>-Alan
> >>
> >>
>
>


More information about the perlbal mailing list