Waiting on replication

[Brian Easley]

Hello,

I'd like to preface by saying that I've been reading the activity on 
this list for about a year, and would would like to thank everyone who 
shares information and provides support on this list.

It seems like I've got MFS working fairly well. I can get the service to 
save a copy of a file, and we're successfully working on converting our 
webapp's php filesystem calls to use a customized MFS API. I've created 
a domain named test and a class named test that has a minimum 
replication count of 1.

I've got two trackers setup successfully and a single 13 disk fileserver 
running mogstored. Do I need more then one fileserver for testing?

The problem comes in after the file is successfully stored for the first 
copy. Here is what happens in command line:

# mogtool --trackers=10.10.10.6:6001,10.10.10.5:6001 --domain=test 
--bigfile inject "CentOS-5.0-i386-bin-1of6.iso" 9108 --concurrent=10 
-overwrite --class=test
chunk 9108,1: 0650b369fe27eea4da74b91c9e7fa556, len = 67108864
Spawned child 27202 to deal with chunk number 1.
chunk 9108,2: 8467daa5e96d28f7fadbdd3509cc10fa, len = 67108864
Spawned child 27203 to deal with chunk number 2.
        chunk 1 saved in 1.03 seconds.complete]
chunk 9108,3: fb056e133949fe862d774dbc5046000a, len = 67108864
Child 27202 successfully finished with chunk 1.
Spawned child 27204 to deal with chunk number 3.
        chunk 2 saved in 0.90 seconds.complete]
chunk 9108,4: 472bfb08dabccfe5774f97cc2c341cf9, len = 67108864
Child 27203 successfully finished with chunk 2.
Spawned child 27205 to deal with chunk number 4.
        chunk 3 saved in 0.87 seconds.complete]
chunk 9108,5: c3925144b01ea80338ac1f83a1a84cf5, len = 67108864
Child 27204 successfully finished with chunk 3.
Spawned child 27206 to deal with chunk number 5.
        chunk 4 saved in 0.97 seconds.complete]
chunk 9108,6: ddee6e9600cb5d50e467e7da38258e89, len = 67108864
Child 27205 successfully finished with chunk 4.
Spawned child 27220 to deal with chunk number 6.
        chunk 5 saved in 1.03 seconds.complete]
chunk 9108,7: 893be1558fd09898c42939b1b2b8f7aa, len = 67108864
Child 27206 successfully finished with chunk 5.
Spawned child 27221 to deal with chunk number 7.
        chunk 6 saved in 1.08 seconds.complete]
chunk 9108,8: 0d4785963b957a8b6522e6ca62a58cc3, len = 67108864
Child 27220 successfully finished with chunk 6.
Spawned child 27222 to deal with chunk number 8.
        chunk 7 saved in 1.02 seconds.complete]
chunk 9108,9: 160dbcec10500e1cefe355558ad24fdd, len = 67108864
Child 27221 successfully finished with chunk 7.
Spawned child 27223 to deal with chunk number 9.
chunk 9108,10: fe4875c855d1e5553b08c643ac17f045, len = 52004864
Spawned child 27224 to deal with chunk number 10.
        chunk 8 saved in 1.77 seconds.
        chunk 10 saved in 1.10 seconds.
Child 27222 successfully finished with chunk 8.
        chunk 9 saved in 1.67 seconds.
Child 27224 successfully finished with chunk 10.
Child 27223 successfully finished with chunk 9.
Beginning replication wait: 1 2 3 4 5 6 7 8 9 10
Beginning replication wait: 1 2 3 4 5 6 7 8 9 10
Beginning replication wait: 1 2 3 4 5 6 7 8 9 10
Beginning replication wait: 1 2 3 4 5 6 7 8 9 10
Beginning replication wait: 1 2 3 4 5 6 7 8 9 10
Beginning replication wait: 1 2 3 4 5 6 7 8 9 10
Beginning replication wait: 1 2 3 4 5 6 7 8 9 10

[root at fs01 ~]# mogtool --trackers=10.10.10.6:6001,10.10.10.5:6001 
--domain=test --bigfile --concurrent=10 -overwrite --class=test inject 
"CentOS-5.0-i386-bin-1of6.iso" 9108 --concurrent=10 -overwrite --class=test
chunk 9108,1: 0650b369fe27eea4da74b91c9e7fa556, len = 67108864
Spawned child 27266 to deal with chunk number 1.
chunk 9108,2: 8467daa5e96d28f7fadbdd3509cc10fa, len = 67108864
Spawned child 27267 to deal with chunk number 2.
        chunk 1 saved in 1.02 seconds.complete]
chunk 9108,3: fb056e133949fe862d774dbc5046000a, len = 67108864
Child 27266 successfully finished with chunk 1.
Spawned child 27268 to deal with chunk number 3.
        chunk 2 saved in 0.91 seconds.complete]
chunk 9108,4: 472bfb08dabccfe5774f97cc2c341cf9, len = 67108864
Child 27267 successfully finished with chunk 2.
Spawned child 27269 to deal with chunk number 4.
chunk 9108,5: c3925144b01ea80338ac1f83a1a84cf5, len = 67108864
Spawned child 27270 to deal with chunk number 5.
        chunk 4 saved in 1.06 seconds.complete]
chunk 9108,6: ddee6e9600cb5d50e467e7da38258e89, len = 67108864
Child 27269 successfully finished with chunk 4.
Spawned child 27271 to deal with chunk number 6.
        chunk 3 saved in 2.90 seconds.complete]
chunk 9108,7: 893be1558fd09898c42939b1b2b8f7aa, len = 67108864
        chunk 5 saved in 1.65 seconds.
Child 27268 successfully finished with chunk 3.
Spawned child 27272 to deal with chunk number 7.
        chunk 6 saved in 1.00 seconds.complete]
chunk 9108,8: 0d4785963b957a8b6522e6ca62a58cc3, len = 67108864
Child 27270 successfully finished with chunk 5.
Child 27271 successfully finished with chunk 6.
        chunk 7 saved in 0.79 seconds.
Spawned child 27286 to deal with chunk number 8.
chunk 9108,9: 160dbcec10500e1cefe355558ad24fdd, len = 67108864
Child 27272 successfully finished with chunk 7.
        chunk 8 saved in 0.83 seconds.
Spawned child 27287 to deal with chunk number 9.
chunk 9108,10: fe4875c855d1e5553b08c643ac17f045, len = 52004864
Child 27286 successfully finished with chunk 8.
Spawned child 27288 to deal with chunk number 10.
        chunk 10 saved in 0.85 seconds.
        chunk 9 saved in 1.70 seconds.
Child 27288 successfully finished with chunk 10.
Child 27287 successfully finished with chunk 9.
Beginning replication wait: 1 2 3 4 5 6 7 8 9 10
Beginning replication wait: 1 2 3 4 5 6 7 8 9 10
Beginning replication wait: 1 2 3 4 5 6 7 8 9 10




The replication count continues eternally.

Two questions:

1) Why is it trying to replicate when the class used has a max 
replication count of 1?
2) Why is it hanging on replication anyways?

Any help would be greatly appreciated.

-Brian