Parallel injects

[Michael A. Toth]

Hi there,

(Sorry for my bad English.) I built a testing environment for
mogilefs, I created a master-slave replicated MySQL-5 databases, two
trackers, four hosts (mogstored) and four devices too, and mogilefs
compiled from svn r1156. I would like to test parallel file injects,
so i wrote a very simple shell script.

#!/bin/bash

n=0

while [ 1 ]; do

  k=$(ps aux | grep '/usr/bin/mogtool'|wc -l)

  [ $k -ge 5 ] && continue

  for j in `seq 1 $((5 - $k))`; do
    n=$((n + 1))
    mogtool inject /testfile.txt test-$n.txt >/dev/null 2>&1 &
  done

done

This executes always min. five inject processes. When I run this
script, and watch process list, approximately 30-35 seconds I see 3 or
4 zombie mogtool processes; and the other procs will never stops. I
checked these procs via strace, and I saw:

7096  --- SIGCHLD (Child exited) @ 0 (0) ---
7096  select(0, NULL, NULL, NULL, {0, 21000}) = 0 (Timeout)
7096  waitpid(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG) = 29678
7096  send(4, "get_paths
domain=testdomain&noverify=1&key=test-16.txt\r\n", 58, MSG_NOSIGNAL) =
58
7096  select(8, [4], NULL, NULL, {3, 0}) = 1 (in [4], left {2, 997000})
7096  read(4, "ERR unknown_key unknown_key\r\n", 4096) = 29
7096  write(1, "Error: reaped child 29678 for chunk 1 but no paths
exist... Retrying...\n", 72) = 72
7096  clone(child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0xb7d87928) = 29685
7096  write(1, "Spawned child 29685 to deal with chunk number 1.\n", 49) = 49
7096  waitpid(-1, 0xbfe23d58, WNOHANG)  = 0
7096  select(0, NULL, NULL, NULL, {0, 100000} <unfinished ...>

(this happens to again for ever, or until I kill)

The machines are Ubuntu Dapper i386 with own kernel-2.6.23, all
depends of mogilefs Perl modules built from cpan, and I prefer Perlbal
for trackers.

Have you got any idea? Please let me if you need more describe.

Regards,
Michael A. Toth