Best architecture

Brad Fitzpatrick brad at danga.com
Tue Jul 10 18:22:35 UTC 2007


Multiple gearmands is for both load balancing and high availability.  No
single point of failure.

Where are you not seeing enough resources used where you'd like more?
You running enough workers?


On Tue, 10 Jul 2007, Matt Sergeant wrote:

> Our current gearman setup doesn't seem to be efficiently using all
> resources available.
>
> We're using it inside qpsmtpd to scan emails (which is slow). Mostly
> it works fine, but we wish it used more resources.
>
> Our current architecture is as follows:
>
> BEGIN
> 4 mail servers running qpsmtpd and scan workers (a simple forking
> server for the workers).
>
> Gearmand running on a "monitor" server and another "spare" server,
> and also on every mail server. I argued against this setup but
> anything else caused too many timeouts.
>
> Workers also run on "monitor" and "spare".
>
> Workers and clients on the mail servers connect to the same
> gearmand's: localhost, monitor, spare.
>
> Workers on "monitor" and "spare" connect to gearmand on localhost only.
> END
>
> Yes it's horrible, and I didn't design it. The ops guy just tried
> running things a few different places until he got something that
> kinda worked - I guess that's the beauty of gearman is that you can
> just bring servers into action wherever and it just keeps running.
>
> So suggestions of better architectures would be welcome.
>
> What's the reason for having clients talk to multiple gearmand's? In
> my original plan I was going to just have gearmand on "monitor" and
> distribute work to all the workers, but that didn't seem to scale
> very well (we're scanning about 90 mails/sec).
>
> Matt.
>
> PS: First Post!!!
>
> _______________________________________________
> Gearman mailing list
> Gearman at lists.danga.com
> http://lists.danga.com/mailman/listinfo/gearman
>
>


More information about the Gearman mailing list