Bizarre perlbal problem

Dormando dormando at rydia.net
Fri Nov 18 10:00:37 PST 2005


I don't think so...

Our backend is just direct connections to PHP running behind either 
apache1 or apache2. As long as the remote user's independent requests go 
down and come back from the same server it should work. IE; If user A 
sends a request down, it just needs to get the reply to that request 
back. If user A sends a request down, but gets the response headers from 
user B, we'd get what we're seeing now.

There's no reliance on the same user hitting the same exact backend 
connection for multiple requests. Before we put the LB in there was no 
keepalive at all, and users would hit one of 80 servers at random.

-Alan

Brad Fitzpatrick wrote:
> Does your backend's session management assuming that for a given
> connection, the same user will always be on that connection?  What backend
> do you use, btw?
> 
> If so, that's your problem, since Perlbal mixes up requests and
> connections.  (on purpose... it's a huge optimization)
> 
> - Brad
> 
> 
> On Fri, 18 Nov 2005, Dormando wrote:
> 
> 
>>(sorry, couldn't resist :P)
>>
>>We got HTTP 1.0 keepalives working (for the most part) late in the day
>>yesterday. Overnight we witnessed a pretty bad glitch where our users
>>would randomly get other user's site cookies and become logged in as
>>someone else.
>>
>>It happened in a small percentage of users, but turning off the backend
>>keepalives seems to have removed the issue. We're still investigating on
>>our end, but I'm having a hard time even speculating how that happened;
>>was perlbal sending back responses from clients other than the
>>requestor? Any insight?
>>
>>Other things that came to mind:
>>  - It's possible sometimes we return an improper content-length.
>>  - If a client connection closes before reading any data back from its
>>connection, does perlbal always junk the backend request before reusing it?
>>
>>I do have a weird routing setup in order to have two perlbal processes
>>running on the same IP address, but since that's just on the frontend
>>and only for incoming requests, I can't see how the responses would get
>>switched... If they were, it'd happen a lot more often than with what we
>>saw.
>>
>>Thanks,
>>-Alan
>>
>>


More information about the perlbal mailing list