recommended number of listener_jobs

[Mark Smith]

Very strange... we regularly have a lot of connections -- probably
hundreds -- coming in from all over the internal network.

When you run your script fork_and_work, you should telnet to your mogilefsd
(port 7001 by default) and do this:

-----------------------------------------
lj at grimace:~$ telnet sto1 7001

Trying 10.0.0.81...
Connected to sto1.lj.
Escape character is '^]'.

!stats

uptime 1294794
pending_queries 0
processing_queries 0
bored_queryworkers 30
queries 124050233
.
-----------------------------------------

Note I typed !stats, and it gave me the output.  (Spacing added for
clarification.)

What you're going to be looking for is it saying that there's a lot of
pending_queries, perhaps?  That would indicate that it can't keep up with
the traffic you're sending.  (But it should never refuse a connection!)

Does the machine you're using have epoll or kqueue?  (You're using a FC3
box, I believe... 2.6 kernel I expect?  So you'd have epoll.)

You can also use another command, !recent, to see the recently executed
commands and how long they took.  Perhaps something's gumming up the works?

-Mark

On Wed, Apr 06, 2005, Robert Rogers wrote:
> Thanks!
> 
> The reason I ask about this is that while benchmarking MogileFS, I've found that when forking off requests for any more than 29 - 30 child processes the mogilefs::backend starts failing to connect and throws this error:
> 
> MogileFS::Backend: couldn't connect to mogilefsd backend at /usr/lib/perl5/site_perl/5.8.5/MogileFS.pm line 91
> 
> Specifically, I've got a basic fork_and_work script which forks off a number of child processes than runs thru a specified number of runs.
> 
> All looks good upto children < 30. After 30 mogilefs starts rejecting requests(??).  Is this weird?
> 
> Thx again.
> 
> Rob
> On Wednesday, April 06, 2005, at 10:04AM, Brad Fitzpatrick <brad at danga.com> wrote:
> 
> >5 or 10 would probably work.  We should make it automatic with a max cap
> >of like 50 which if it reaches means it's the database that's loaded, and
> >not the listener jobs.
> >
> >Here's our config:
> >
> ># cat /etc/mogilefs/mogilefsd.conf
> >daemonize = 1
> >db_dsn = DBI:mysql:mogilefs:host=10.0.0.48
> >db_user = mog
> >db_pass = XXXXXXXXX
> >conf_port = 7001
> >listener_jobs = 30
> >delete_jobs = 1
> >replicate_jobs = 5
> >mog_root = /mnt/mogilefs
> >reaper_jobs = 0
> >
> >
> >Though I'm not sure why we have mog_root defined... think that's old
> >historic crap that we don't use anymore now that we're using HTTP mode.
> >
> >- Brad
> >
> >
> >On Wed, 6 Apr 2005, Rob Rogers wrote:
> >
> >> Hi,
> >>
> >> Anybody have suggestions for number of listener_jobs for MogileFSD.
> >> (or for that matter a good .conf file for Mogilefsd)
> >>
> >> Thx.
> >>
> >> Rob
> >>
> >>
> >
> >


--
Mark Smith
junior at danga.com