recommended number of listener_jobs
marksmith at danga.com
Wed Apr 6 11:17:53 PDT 2005
Very strange... we regularly have a lot of connections -- probably
hundreds -- coming in from all over the internal network.
When you run your script fork_and_work, you should telnet to your mogilefsd
(port 7001 by default) and do this:
lj at grimace:~$ telnet sto1 7001
Connected to sto1.lj.
Escape character is '^]'.
Note I typed !stats, and it gave me the output. (Spacing added for
What you're going to be looking for is it saying that there's a lot of
pending_queries, perhaps? That would indicate that it can't keep up with
the traffic you're sending. (But it should never refuse a connection!)
Does the machine you're using have epoll or kqueue? (You're using a FC3
box, I believe... 2.6 kernel I expect? So you'd have epoll.)
You can also use another command, !recent, to see the recently executed
commands and how long they took. Perhaps something's gumming up the works?
On Wed, Apr 06, 2005, Robert Rogers wrote:
> The reason I ask about this is that while benchmarking MogileFS, I've found that when forking off requests for any more than 29 - 30 child processes the mogilefs::backend starts failing to connect and throws this error:
> MogileFS::Backend: couldn't connect to mogilefsd backend at /usr/lib/perl5/site_perl/5.8.5/MogileFS.pm line 91
> Specifically, I've got a basic fork_and_work script which forks off a number of child processes than runs thru a specified number of runs.
> All looks good upto children < 30. After 30 mogilefs starts rejecting requests(??). Is this weird?
> Thx again.
> On Wednesday, April 06, 2005, at 10:04AM, Brad Fitzpatrick <brad at danga.com> wrote:
> >5 or 10 would probably work. We should make it automatic with a max cap
> >of like 50 which if it reaches means it's the database that's loaded, and
> >not the listener jobs.
> >Here's our config:
> ># cat /etc/mogilefs/mogilefsd.conf
> >daemonize = 1
> >db_dsn = DBI:mysql:mogilefs:host=10.0.0.48
> >db_user = mog
> >db_pass = XXXXXXXXX
> >conf_port = 7001
> >listener_jobs = 30
> >delete_jobs = 1
> >replicate_jobs = 5
> >mog_root = /mnt/mogilefs
> >reaper_jobs = 0
> >Though I'm not sure why we have mog_root defined... think that's old
> >historic crap that we don't use anymore now that we're using HTTP mode.
> >- Brad
> >On Wed, 6 Apr 2005, Rob Rogers wrote:
> >> Hi,
> >> Anybody have suggestions for number of listener_jobs for MogileFSD.
> >> (or for that matter a good .conf file for Mogilefsd)
> >> Thx.
> >> Rob
junior at danga.com
More information about the mogilefs