Perl's autoflush() and Gearman workers
goffinet at yahoo-inc.com
Wed Jan 9 07:21:46 UTC 2008
Yeah I found out that gearman is super slow :-D Will be working to see
if we can improve. This is only when you do normal job types in
parallel, not background jobs. We have been benchmarking more workers
== greater latency for X tasks. So if i submitted 25 tasks with 4
workers running idle (that just returned the number 1) and I started
10 more workers, latency with gearman increases dramatically. Over
100ms in some cases.
I am interested in the gearman c version and will be doing tests using
San Francisco, CA
On Jan 8, 2008, at 4:22 PM, Joe Stump wrote:
> So I've been digging and digging and would love for someone who
> actually works on Gearman to read this message and possibly reply
> back to me. I've been working with Chris G. from Yahoo! on a PHP
> library called Net_Gearman (which I've been spamming the list with
> updates about).
> I've finally got my PHP code running as fast as the Perl one does
> with a single worker running a few tasks. The problem, however, is
> that when I add workers with the Perl code the time it takes the
> tasks to run halves (which is obvious as it's running them in
> parallel). The same is exactly the opposite with PHP. It goes from
> 11ms to run 25 tasks with a single worker to taking 50ms. I've
> verified that each worker is, in fact, getting an equal number of
> jobs, etc.
> The problem is that a socket_select() waits around on the client
> side for anywhere from 30 and 45ms waiting for data when I know the
> worker has already written data back to gearmand.
> I have a nagging feeling this has something to do with Perl's
> autoflush on it's IO::Socket, but I can't for the live of me figure
> out what $| = 0 and $| = 1 actually does under the hood to "auto
> flush" the socket. My thinking is that the socket isn't autoflushing
> in my PHP code and, as a result, isn't sending data as soon as I
> write it to the socket from the worker.
> Gearman mailing list
> Gearman at lists.danga.com
More information about the Gearman