Size limit for tasks?
jsteinert at sixapart.com
Mon Apr 21 20:23:06 UTC 2008
I fixed this in trunk around r337 for the perl client. There was
an implicit job size limitation that I removed.
And then around r339 I made 'large' jobs a lot faster.
If you can, please try using trunk/HEAD to see if it removes your
problem. I'll see what I can do to usher along a release.
On Mon, Apr 21, 2008 at 09:50:27AM -0700, Paul Goracke wrote:
> On Apr 21, 2008, at 9:33 AM, Joe Stump wrote:
>> On Apr 21, 2008, at 8:39 AM, Paul Goracke wrote:
>>> Is there a known size limitation for submitted tasks? I'm running
>>> into problems with tasks failing if they exceed a certain size. To
>>> make matters more frustrating, the size doesn't seem to be a
>>> constant: At first it seemed to be the magic 64k, but that number
>>> has varied as I've been testing. One of the strange aspects of this
>>> is that I'm only sending out data I have received from other
>>> workers--so Gearman is handling apparently arbitrarily-sized task
>>> results, but not task params.
>> When you say "failing" do you mean a hard fail or that they fail when
>> things are unserialized?
> It's a hard fail. do_task() returns undef; running in a taskset will
> call a on_retry handler the appropriate number of times, then on_fail.
>> With my PHP client/worker code I was running into UTF8 problems when
>> using strlen() (which isn't unicode aware). If your results can be in
>> UTF8 and the Perl code isn't accounting for that then things could be
>> blowing up for that reason. Again, this was a PHP-only problem I ran
>> into ...
> That was one of my initial suspects; the "real product" is dealing in
> UTF8 strings within Perl objects serialized via Storable's nfreeze() for
> task submission. (The length() call in Perl is unicode-aware, but is not
> used in my product, just these test scripts.) But the test I attached
> still fails just by creating a string of 'x' characters yet properly
> handles the same characters in a shorter string, so I have ruled it
> out--at least for this issue.
> Very preliminary debugging on the server side seems to indicate the
> server _is_ getting all of the task but then for some reason is being
> failed off the job queue--I have never seen it reach the worker.
> Curiouser and curiouser.
> Gearman mailing list
> Gearman at lists.danga.com
More information about the Gearman