recommended number of listener_jobs

[Mark Smith]

!help shows you the list of commands available.  There is no way to quit,
just close the connection... :)

This is a very strange problem.  Like your other problem with Linux::AIO
returning -1 but not setting $!.

I don't have an FC3 box here to play around on and see if I can duplicate
these problems, so I don't know what to suggest.

Idea: start up mogilefsd, then find its pid number and:

# strace -p <pid> 2>&1 | egrep "accept|epoll"

Then run your fork_and_work script.  Notably, I'm curious about any errors
that accept() runs into, or any errors you see with the epoll calls.  (And
to make sure that it's using epoll!)

> Mark
> 
> You're right - FC3.  
> 
> I've been checking the stats : and nothing seems to be out of order.
> 
> Any suggestions on trouble shooting. 
> 
> Also, any other commands, especially how to quit :-)
> 
> -Rob
> On Wednesday, April 06, 2005, at 11:17AM, Mark Smith <marksmith at danga.com> wrote:
> 
> >Very strange... we regularly have a lot of connections -- probably
> >hundreds -- coming in from all over the internal network.
> >
> >When you run your script fork_and_work, you should telnet to your mogilefsd
> >(port 7001 by default) and do this:
> >
> >-----------------------------------------
> >lj at grimace:~$ telnet sto1 7001
> >
> >Trying 10.0.0.81...
> >Connected to sto1.lj.
> >Escape character is '^]'.
> >
> >!stats
> >
> >uptime 1294794
> >pending_queries 0
> >processing_queries 0
> >bored_queryworkers 30
> >queries 124050233
> >.
> >-----------------------------------------
> >
> >Note I typed !stats, and it gave me the output.  (Spacing added for
> >clarification.)
> >
> >What you're going to be looking for is it saying that there's a lot of
> >pending_queries, perhaps?  That would indicate that it can't keep up with
> >the traffic you're sending.  (But it should never refuse a connection!)
> >
> >Does the machine you're using have epoll or kqueue?  (You're using a FC3
> >box, I believe... 2.6 kernel I expect?  So you'd have epoll.)
> >
> >You can also use another command, !recent, to see the recently executed
> >commands and how long they took.  Perhaps something's gumming up the works?
> >
> >-Mark
> >
> >On Wed, Apr 06, 2005, Robert Rogers wrote:
> >> Thanks!
> >> 
> >> The reason I ask about this is that while benchmarking MogileFS, I've found that when forking off requests for any more than 29 - 30 child processes the mogilefs::backend starts failing to connect and throws this error:
> >> 
> >> MogileFS::Backend: couldn't connect to mogilefsd backend at /usr/lib/perl5/site_perl/5.8.5/MogileFS.pm line 91
> >> 
> >> Specifically, I've got a basic fork_and_work script which forks off a number of child processes than runs thru a specified number of runs.
> >> 
> >> All looks good upto children < 30. After 30 mogilefs starts rejecting requests(??).  Is this weird?
> >> 
> >> Thx again.
> >> 
> >> Rob
> >> On Wednesday, April 06, 2005, at 10:04AM, Brad Fitzpatrick <brad at danga.com> wrote:
> >> 
> >> >5 or 10 would probably work.  We should make it automatic with a max cap
> >> >of like 50 which if it reaches means it's the database that's loaded, and
> >> >not the listener jobs.
> >> >
> >> >Here's our config:
> >> >
> >> ># cat /etc/mogilefs/mogilefsd.conf
> >> >daemonize = 1
> >> >db_dsn = DBI:mysql:mogilefs:host=10.0.0.48
> >> >db_user = mog
> >> >db_pass = XXXXXXXXX
> >> >conf_port = 7001
> >> >listener_jobs = 30
> >> >delete_jobs = 1
> >> >replicate_jobs = 5
> >> >mog_root = /mnt/mogilefs
> >> >reaper_jobs = 0
> >> >
> >> >
> >> >Though I'm not sure why we have mog_root defined... think that's old
> >> >historic crap that we don't use anymore now that we're using HTTP mode.
> >> >
> >> >- Brad
> >> >
> >> >
> >> >On Wed, 6 Apr 2005, Rob Rogers wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> Anybody have suggestions for number of listener_jobs for MogileFSD.
> >> >> (or for that matter a good .conf file for Mogilefsd)
> >> >>
> >> >> Thx.
> >> >>
> >> >> Rob
> >> >>
> >> >>
> >> >
> >> >
> >
> >
> >--
> >Mark Smith
> >junior at danga.com
> >
> >


--
Mark Smith
junior at danga.com