fair load balancing?
moses at collaborativedrug.com
Mon Jun 23 22:33:58 UTC 2008
On Jun 23, 2008, at 2:50 PM, dormando wrote:
> Backend goes back to the pool. Options request is sent once per
> established backend connection; so if you connect to a backend and
> it 500 times, you'd have one OPTIONS request and 500 normal requests.
And, because of this, I don't think the proposed solution works for
the "long-running mongrel request" problem with persist_backend on.
Also, if the OPTIONS request actually is served by the rails stack,
then the OPTIONS request hangs just like a normal request would, and
you see the same undesirable behavior (a normally short request taking
a long time because it's waiting for a response from a mongrel that is
serving a long-running Rails request). I think in order for this to
work as intended you need both persist_backend off (which does mean
double the number of requests to your mongrels, but probably that's
not a huge deal) and you need to use the mongrel_rails option --num-
processors 1, which will limit the number of worker threads operating
in the mongrel to 1 at a time. Even when I do this, though, I'm still
able to get the web stack to hang without too much effort by just
reloading a page many times in succession very quickly (albeit with a
very fast network connection to the server). It's entirely possible
that I'm still missing something here, though. By chance I'm going to
get to talk to Adam in person tonight, so I hope to sort these details
out with him then, and I will report back to the list.
I think a while back in this thread someone said this was not a
Perlbal problem, but a mongrel problem, but it would be more accurate
to say this is a Rails problem. Mongrel can handle concurrent requests
just fine. Rails imposes the single-thread of execution mutex lock.
This could also be viewed as an application-level problem (i.e. why
are your requests taking longer than a couple seconds anyway, and if
they must, why not offload the actual processing to a background job
and have the web browser poll until the background job is done), which
is sort of true. However, it would be easier if the load balancer
could just handle this for us, as that would also insulate us from
unanticipated slow requests as well.
More information about the perlbal