MogileFS Excessive CPU Utilization on Idle Server

[Erik Osterman]

I think the problem is we did not explicitly specify "old_repl_compat 0" 
in mogilefsd.conf

Erik Osterman


Erik Osterman wrote:
> MogileFS has been consistently using an unusual amount of CPU. Even 
> with our application entirely turned off, MogileFS is busy doing 
> something. I am new to the whole MogileFS app, so it could be entirely 
> likely that I've misconfigured something.
>
>  837 mysql     15   0  142m  28m 4024 S   62  2.9   1804:36 mysqld
> 18268 mogile    16   0 21076  10m 1540 R   56  1.0   1034:28 mogilefsd
> 18216 mogile    15   0 21076  10m 1732 S   14  1.0 207:51.28 mogilefsd
> 26967 mogile    15   0 21076 9204 1364 R   14  0.9 680:22.23 mogilefsd
>
>
> An strace shows..
>
> poll([{fd=4, events=POLLIN|POLLPRI}], 1, 0) = 0
> write(4, "1\0\0\0\3SELECT 
> RELEASE_LOCK(\'mgfs:fid:288598:replicate\')", 53) = 53
> read(4, 
> "\1\0\0\1\1?\0\0\2\3def\0\0\0)RELEASE_LOCK(\'mgfs:fid:288598:replicate\')\0\f?\0\1\0\0\0\10\200\0\0\0\0\5\0\0\3\376\0\0\2\0\2\0\0\4\0011\5\0\0\5\376\0\0\2\0", 
> 16384) = 96
> time(NULL)                              = 1181588819
> time(NULL)                              = 1181588819
> time(NULL)                              = 1181588819
> write(21, ":still_alive\r\n", 14)       = 14
> poll([{fd=4, events=POLLIN|POLLPRI}], 1, 0) = 0
> write(4, "i\0\0\0\3SELECT fid FROM file WHERE dmid=\'1\' AND 
> classid=\'4\' AND devcount = \'1\' AND length IS NOT NULL LIMIT 1000", 
> 109) = 109
> read(4, 
> "\1\0\0\1\1,\0\0\2\3def\10mogilefs\4file\4file\3fid\3fid\f?\0\n\0\0\0\3#P\0\0\0\5\0\0\3\376\0\0\2\0\7\0\0\4\006252333\7\0\0\5\006252799\7\0\0\6\006255062\7\0\0\7\006341306\7\0\0\10\006341511\7\0\0\t\006342276\7\0\0\n\006342284\7\0\0\v\006342285\5\0\0\f\376\0\0\2\0", 
> 16384) = 159
> time(NULL)                              = 1181588819
> time(NULL)                              = 1181588819
> poll([{fd=4, events=POLLIN|POLLPRI}], 1, 0) = 0
> write(4, "i\0\0\0\3SELECT fid FROM file WHERE dmid=\'1\' AND 
> classid=\'1\' AND devcount = \'1\' AND length IS NOT NULL LIMIT 1000", 
> 109) = 109
> read(4, 
> "\1\0\0\1\1,\0\0\2\3def\10mogilefs\4file\4file\3fid\3fid\f?\0\n\0\0\0\3#P\0\0\0\5\0\0\3\376\0\0\2\0\7\0\0\4\006250518\7\0\0\5\006250520\7\0\0\6\006250528\7\0\0\7\006250653\7\0\0\10\006250656\7\0\0\t\006250777\7\0\0\n\006250778\7\0\0\v\006250780\7\0\0\f\006250781\7\0\0\r\006250789\7\0\0\16\006250790\7\0\0\17\006250791\7\0\0\20\0062"..., 
> 16384) = 11071
> time(NULL)                              = 1181588819
> time(NULL)                              = 1181588819
> poll([{fd=4, events=POLLIN|POLLPRI}], 1, 0) = 0
> write(4, "i\0\0\0\3SELECT fid FROM file WHERE dmid=\'1\' AND 
> classid=\'0\' AND devcount = \'1\' AND length IS NOT NULL LIMIT 1000", 
> 109) = 109
> read(4, 
> "\1\0\0\1\1,\0\0\2\3def\10mogilefs\4file\4file\3fid\3fid\f?\0\n\0\0\0\3#P\0\0\0\5\0\0\3\376\0\0\2\0\5\0\0\4\376\0\0\2\0", 
> 16384) = 71
> time(NULL)                              = 1181588819
> time(NULL)                              = 1181588819
> poll([{fd=4, events=POLLIN|POLLPRI}], 1, 0) = 0
> write(4, "i\0\0\0\3SELECT fid FROM file WHERE dmid=\'1\' AND 
> classid=\'3\' AND devcount = \'1\' AND length IS NOT NULL LIMIT 1000", 
> 109) = 109
> read(4, 
> "\1\0\0\1\1,\0\0\2\3def\10mogilefs\4file\4file\3fid\3fid\f?\0\n\0\0\0\3#P\0\0\0\5\0\0\3\376\0\0\2\0\7\0\0\4\006250902\7\0\0\5\006251006\7\0\0\6\006251020\7\0\0\7\006251220\7\0\0\10\006251222\7\0\0\t\006251223\7\0\0\n\006251224\7\0\0\v\006251227\7\0\0\f\006251229\7\0\0\r\006251230\7\0\0\16\006251231\7\0\0\17\006251233\7\0\0\20\0062"..., 
> 16384) = 951
> time(NULL)                              = 1181588819
> time(NULL)                              = 1181588819
> poll([{fd=4, events=POLLIN|POLLPRI}], 1, 0) = 0
> write(4, "i\0\0\0\3SELECT fid FROM file WHERE dmid=\'1\' AND 
> classid=\'2\' AND devcount = \'1\' AND length IS NOT NULL LIMIT 1000", 
> 109) = 109
> read(4, 
> "\1\0\0\1\1,\0\0\2\3def\10mogilefs\4file\4file\3fid\3fid\f?\0\n\0\0\0\3#P\0\0\0\5\0\0\3\376\0\0\2\0\7\0\0\4\006250965\7\0\0\5\006251499\7\0\0\6\006251510\7\0\0\7\006251511\7\0\0\10\006251512\7\0\0\t\006251513\7\0\0\n\006251524\7\0\0\v\006251525\7\0\0\f\006251526\7\0\0\r\006251572\7\0\0\16\006251613\7\0\0\17\006251627\7\0\0\20\0062"..., 
> 16384) = 11071
>
>
>
> Is this to be expected?
>
> FYI, I am running a relatively recent svn version, as of about 2 weeks 
> ago.
>
>
> Best,
>
> Erik Osterman
>
>
>