Size limit for tasks?

[Jonathan Steinert]

I fixed this in trunk around r337 for the perl client. There was
an implicit job size limitation that I removed.

And then around r339 I made 'large' jobs a lot faster.

If you can, please try using trunk/HEAD to see if it removes your
problem. I'll see what I can do to usher along a release.

--hachi

On Mon, Apr 21, 2008 at 09:50:27AM -0700, Paul Goracke wrote:
> On Apr 21, 2008, at 9:33 AM, Joe Stump wrote:
>
>>
>> On Apr 21, 2008, at 8:39 AM, Paul Goracke wrote:
>>
>>> Is there a known size limitation for submitted tasks? I'm running  
>>> into problems with tasks failing if they exceed a certain size. To  
>>> make matters more frustrating, the size doesn't seem to be a  
>>> constant: At first it seemed to be the magic 64k, but that number  
>>> has varied as I've been testing. One of the strange aspects of this  
>>> is that I'm only sending out data I have received from other  
>>> workers--so Gearman is handling apparently arbitrarily-sized task  
>>> results, but not task params.
>>
>> Paul,
>>
>> When you say "failing" do you mean a hard fail or that they fail when 
>> things are unserialized?
>
> It's a hard fail. do_task() returns undef; running in a taskset will  
> call a on_retry handler the appropriate number of times, then on_fail.
>
>> With my PHP client/worker code I was running into UTF8 problems when  
>> using strlen() (which isn't unicode aware). If your results can be in 
>> UTF8 and the Perl code isn't accounting for that then things could be 
>> blowing up for that reason. Again, this was a PHP-only problem I ran 
>> into ...
>
> That was one of my initial suspects; the "real product" is dealing in  
> UTF8 strings within Perl objects serialized via Storable's nfreeze() for 
> task submission. (The length() call in Perl is unicode-aware, but is not 
> used in my product, just these test scripts.) But the test I attached 
> still fails just by creating a string of 'x' characters yet properly 
> handles the same characters in a shorter string, so I have ruled it 
> out--at least for this issue.
>
> Very preliminary debugging on the server side seems to indicate the  
> server _is_ getting all of the task but then for some reason is being  
> failed off the job queue--I have never seen it reach the worker.  
> Curiouser and curiouser.
>
> pg
>
> _______________________________________________
> Gearman mailing list
> Gearman at lists.danga.com
> http://lists.danga.com/mailman/listinfo/gearman