perlbal use case question
mike.engelhart at gmail.com
Tue Jun 10 18:01:06 UTC 2008
Thanks Nathan -
So I think you have it in a nutshell as to what I'm trying to do and I
also agree with your analysis that it's too many moving parts which is
why I'm a bit stymied how to handle this. If we were using Java or
something the architecture would be much simpler but currently that's
not an option on the table.
Re: Apache - the only reason I thought I'd need to use that would be
for SSL handling - i was under the impression that Perlbal didn't
handle SSL sessions - maybe it does? These have to be signed
certificates (i.e Verisign, etc due our clients requirements).
I like the idea of Perlbal waiting but again not knowing much about
it, it's described as being single threaded so I assumed that it can't
block on a request like that but I guess some asynchronous call could
be made and then return? Should I be looking into the hooks
documentation? I'm not sure where to even start at this point.
On Jun 10, 2008, at 1:45 PM, Nathan Kurz wrote:
> On Tue, Jun 10, 2008 at 8:34 AM, Michael Engelhart
> <mike.engelhart at gmail.com> wrote:
>> I'm looking into Perlbal to solve a particular problem with a new web
>> services platform that we're developing.
> I'm new to Perlbal, but I'll take a stab as I've been looking at using
> it for something similar (aggregating search results from multiple
> shards on a local network).
> First, let me see if I understand correctly what you are trying to do:
> 1) You get a user request, which maps to multiple external requests.
> 2) You make these slow requests and wait for the responses.
> 3) You aggregate these results and return them to the user.
> And you are planning to do this by:
> 1) Perlbal proxies the request to Apache which proxies to your
> 2) Application sends back Perlbal redirect with payload of URLs.
> 3) Perlbal gives URLs to one of a pool of fetchers (or is it one per
> 4) Fetcher returns Perlbal redirect with aggregated XML responses as
> 5) Perlbal redirects back to Apache which proxies to application to
> If this is what you are doing, my instinct is that Perlbal's internal
> redirects would work for you, but your architecture has more moving
> parts than you really need. For example, I'm not sure what role
> Apache is playing for you here, and I'd guess that a very small amount
> of code would allow Perlbal to make and wait for the outside requests
> in a nice non-blocking way. This might just be prejudice on my part,
> as I'd be more comfortable extending Perlbal a small amount rather
> than writing that custom lightweight process.
> My model would probably look more like this:
> 1) Perlbal proxies user request directly to one of a pool of app
> 2) App server does some work and returns a 'custom' redirect and a
> list of URLs.
> 3) Custom Perlbal plugin queues and makes asynchronous external
> 4) Collected responses are redirected to a second pool of app servers.
> 5) Second level app server homogenizes responses and generated
> finished XML.
> The first and second level app servers could be the same pool of
> processes with a different request structure, but barring odd
> circumstances it strikes me as easier and more scalable to keep them
> Or perhaps I misunderstand your goals. It probably could also be
> done using nothing but Perlbal redirects as you describe.
> Hope this helps,
> Nathan Kurz
> nate at verse.com
More information about the perlbal