Multiple Proxy Requests

[Mark Smith]

> I'm new to  Perlbal, and considering using it aggregate the results of
> multiple parallel HTTP requests.   The application would be part of a
> distributed search engine.  A request comes in, and instead of
> proxying it to a single backend, I would make simultaneous requests to
> one server from each of a dozen or so pools.  Then I collate the
> results, and send the processed results back to the original client.
>
> Does this seem like a good fit for Perlbal?  Would the plugin
> architecture allow this sort of thing?

Yeah, you could do this.  You would in effect just spawn a bunch of
Backends and have your plugin hooked in to them so that when they
return from getting the answer they can do the right thing with the
ClientProxy objects.

It would be doable, yes!

> Looking back through the archives, I found a somewhat similar question
> from last year
> (http://lists.danga.com/pipermail/perlbal/2007-July/000537.html) where
> it was suggested to use Gearman.  Does this advice still apply?  It
> feels like there should be some way I can subvert the existing Perlbal
> infrastructure to do this instead.

Well yes, this is probably the canonical way/recommended way, but it
does involve a separate process.  I guess it comes down to what feel
comfortable doing and whether or not you need these jobs being done by
other things too?

Like, if you only ever expect to do the backend processing for HTTP
requests, doing it all in Perlbal works fine.  But if you intend to
have cron jobs or other tasks that require the same logic, it might
make sense to do it outside of Perlbal.  Dig?

Either way, I definitely think it's doable within the Perlbal framework.  :)


-- 
Mark Smith / xb95
smitty at gmail.com