fair load balancing?

[Nathan Kurz]

On Mon, Jun 23, 2008 at 3:50 PM, Mark Smith <smitty at gmail.com> wrote:
> Perlbal does not do 'sticky sessions', which is basically what you're
> asking about here.  When a client and backend have finished with a
> request, the backend goes back into the pool to serve the next request
> in the queue.

Great.  That was the conclusion I came to from looking at the code,
but I wasn't sure if I was missing something.  In my case, the absence
of stickiness counts as a feature!

> Interesting.  So something like:
>
> {internet clients} -> [1+ frontend proxy] -> [1 proxy, 1+ web serving processes]

I'm not sure how to represent it textually, but it will be more like:

{internet clients} ->
    [Apache to FCGI Catalyst] ->
       [1 (of 2) frontend proxies] ->
         [12 (of 24) backend proxies] ->
             [each to 1 (of 8) local search processes].

The non-standard part here is that the front-end proxy will split the
request into 12 (or so) duplicates and aggregate the responses from
the backends.  Each pool of backends represents a shard of the index,
such that we need a response from a member of each pool to complete a
query.

> I'm not TOO sure why you have multiple processes that need to be
> proxied to on the individual machine.  Can't you just do that from the
> initial frontend proxy?  What's the purpose of the two layers?

It's quite possible there is  better solution here.  In my mind, the
second layer is going to help me to distribute the work evenly while
keeping the number of search processes to one per core.   I wanted to
stay with persistent connections, and I didn't see a good way to do
this with direct connections to the processes.
I didn't see how to have multiple front end servers share the
persistent connections fairly while avoiding queuing fast requests
behind slow ones.  But I came up with this model before I understood
verify_backend.  Perhaps it solves this problem too?

Maybe the right thing for me to do would be to run with a single front
end proxy, which would avoid most such problems.  Is that what others
are doing?

Nathan Kurz
nate at verse.com