perlbal use case question

Michael Engelhart mike.engelhart at gmail.com
Tue Jun 10 18:01:06 UTC 2008


Thanks Nathan -

So I think you have it in a nutshell as to what I'm trying to do and I  
also agree with your analysis that it's too many moving parts which is  
why I'm a bit stymied how to handle this.   If  we were using Java or  
something the architecture would be much simpler but currently that's  
not an option on the table.

Re: Apache - the only reason I thought I'd need to use that  would be  
for SSL handling - i was under the impression that Perlbal didn't  
handle SSL sessions - maybe it does?   These have to be signed  
certificates (i.e Verisign, etc due our clients requirements).

I like the idea of Perlbal waiting but again not knowing much about  
it, it's described as being single threaded so I assumed that it can't  
block on a request like that but I guess some asynchronous call could  
be made and then return?     Should I be looking into the hooks  
documentation?    I'm not sure where to even start at this point.

Thanks again

MIke


On Jun 10, 2008, at 1:45 PM, Nathan Kurz wrote:

> On Tue, Jun 10, 2008 at 8:34 AM, Michael Engelhart
> <mike.engelhart at gmail.com> wrote:
>> I'm looking into Perlbal to solve a particular problem with a new web
>> services platform that we're developing.
>
> I'm new to Perlbal, but I'll take a stab as I've been looking at using
> it for something similar (aggregating search results from multiple
> shards on a local network).
>
> First, let me see if I understand correctly what you are trying to do:
> 1) You get a user request, which maps to multiple external requests.
> 2) You make these slow requests and wait for the responses.
> 3) You aggregate these results and return them to the user.
>
> And you are planning to do this by:
> 1) Perlbal proxies the request to Apache which proxies to your  
> application
> 2) Application sends back Perlbal redirect with payload of URLs.
> 3) Perlbal gives URLs to one of a pool of fetchers (or is it one per  
> URL?).
> 4) Fetcher returns Perlbal redirect with aggregated XML responses as  
> payload.
> 5) Perlbal redirects back to Apache which proxies to application to  
> finish.
>
> If this is what you are doing, my instinct is that Perlbal's internal
> redirects would work for you, but your architecture has more moving
> parts than you really need.  For example, I'm not sure what role
> Apache is playing for you here, and I'd guess that a very small amount
> of code would allow Perlbal to make and wait for the outside requests
> in a nice non-blocking way.  This might just be prejudice on my part,
> as I'd be more comfortable extending Perlbal a small amount rather
> than writing that custom lightweight process.
>
> My model would probably look more like this:
> 1) Perlbal proxies user request directly to one of a pool of app  
> servers.
> 2) App server does some work and returns a 'custom' redirect and a  
> list of URLs.
> 3) Custom Perlbal plugin queues and makes asynchronous external  
> requests.
> 4) Collected responses are redirected to a second pool of app servers.
> 5) Second level app server homogenizes responses and generated  
> finished XML.
>
> The first and second level app servers could be the same pool of
> processes with a different request structure, but barring odd
> circumstances it strikes me as easier and more scalable to keep them
> separate.
>
> Or perhaps I misunderstand your goals.    It probably could also be
> done using nothing but Perlbal redirects as you describe.
>
> Hope this helps,
>
> Nathan Kurz
> nate at verse.com



More information about the perlbal mailing list