Trouble setting up mogileFS
Arnoud Vermeer
a.vermeer at freshway.biz
Sun Jan 29 18:48:45 UTC 2006
Hi,
I wasn't having trouble with the creation of the /0/000/000 dirs, but with
the creation of the /var/mogdata/dev$nr
After I created the following, the /0/000/000 came automaticly, so that
wasn't the case with me.
I agree that the documentation is spread very thin, that's a mayor
bottleneck for such a great project.
I'd love to contribute to the documentation, but I don't have time to be the
main cheerleader sorry :)
Thanks for your help.
Arnoud Vermeer
On 1/29/06, Brad Fitzpatrick <brad at danga.com> wrote:
>
> The problem that Paul Hurley was having here,
> http://lists.danga.com/pipermail/mogilefs/2006-January/000244.html
>
> , with the missing 0/000/000 directories, is a broken Linux::AIO.
>
> But you always have to create the /var/mogdata/dev$n directories.
>
> Use this script to auto-create/mount all your /var/mogdata directories:
>
> http://cvs.danga.com/browse.cgi/wcmtools/mogilefs/server/mogautomount?rev=1.2
>
> (see the docs at bottom, or run "perldoc" on it)
>
> It works if you put labels on your Mogile filesystems, which is *highly
> recommended*.
>
> Is anybody on this list passionate/energetic/bored enough to take up
> maintenance of MogileFS docs and lead cheerleading? I'm doing a terrible
> job. We are trying to get it into Debian here, and are making progress on
> everything required for that, but our website, for instance, is pretty
> bad. In particular, we need a high-level overview and installation guide.
>
> - Brad
>
>
> On Sun, 29 Jan 2006, Arnoud Vermeer wrote:
>
> > I've got Linux:AIO installed via cpan install Linux::AIO, but I've still
> had
> > the error, that's why I guessed there was something wrong :)
> >
> > test results Debian Sarge (i386)
> > IO-AIO-1.71# make test
> > cp autoconf.pm blib/lib/IO/autoconf.pm
> > cp AIO.pm blib/lib/IO/AIO.pm
> > /usr/bin/perl /usr/share/perl/5.8/ExtUtils/xsubpp -typemap
> > /usr/share/perl/5.8/ExtUtils/typemap AIO.xs > AIO.xsc && mv AIO.xsc
> AIO.c
> > cc -c -D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBIAN
> > -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE
> > -D_FILE_OFFSET_BITS=64 -O2 -DVERSION=\"1.71\" -DXS_VERSION=\"1.71\"
> -fPIC
> > "-I/usr/lib/perl/5.8/CORE" AIO.c
> > Running Mkbootstrap for IO::AIO ()
> > chmod 644 AIO.bs
> > rm -f blib/arch/auto/IO/AIO/AIO.so
> > LD_RUN_PATH="" cc -shared -L/usr/local/lib AIO.o -o
> > blib/arch/auto/IO/AIO/AIO.so -lpthread -lrt
> > chmod 755 blib/arch/auto/IO/AIO/AIO.so
> > cp AIO.bs blib/arch/auto/IO/AIO/AIO.bs
> > chmod 644 blib/arch/auto/IO/AIO/AIO.bs
> > PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e"
> > "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
> > t/00_load.......ok
> > t/01_stat.......ok
> > t/02_read.......ok
> > t/03_errors.....ok
> > t/04_fork.......ok
> > t/05_readdir....ok
> > All tests successful.
> > Files=6, Tests=44, 1 wallclock secs ( 0.45 cusr + 0.05 csys = 0.50CPU)
> >
> > Linux-AIO-1.8# make test
> > PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e"
> > "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
> > t/00_load......ok
> > t/01_stat......ok
> > t/02_read......ok
> > t/03_errors....ok
> > All tests successful.
> > Files=4, Tests=26, 1 wallclock secs ( 0.42 cusr + 0.04 csys = 0.46CPU)
> >
> > -------
> > IO-AIO under FreeBSD amd64
> >
> > make test
> > cp autoconf.pm blib/lib/IO/autoconf.pm
> > cp AIO.pm blib/lib/IO/AIO.pm
> > /usr/bin/perl /usr/local/lib/perl5/5.8.7/ExtUtils/xsubpp -typemap
> > /usr/local/lib/perl5/5.8.7/ExtUtils/typemap AIO.xs > AIO.xsc && mv
> AIO.xsc
> > AIO.c
> > cc -c -DAPPLLIB_EXP="/usr/local/lib/perl5/5.8.7/BSDPAN"
> -DHAS_FPSETMASK
> > -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -I/usr/local/include -O
> > -pipe -DVERSION=\"1.71\" -DXS_VERSION=\"1.71\" -DPIC -fPIC
> > "-I/usr/local/lib/perl5/5.8.7/mach/CORE" AIO.c
> > Running Mkbootstrap for IO::AIO ()
> > chmod 644 AIO.bs
> > rm -f blib/arch/auto/IO/AIO/AIO.so
> > LD_RUN_PATH="/usr/lib" cc -shared -L/usr/local/lib AIO.o -o
> > blib/arch/auto/IO/AIO/AIO.so -lpthread
> > chmod 755 blib/arch/auto/IO/AIO/AIO.so
> > cp AIO.bs blib/arch/auto/IO/AIO/AIO.bs
> > chmod 644 blib/arch/auto/IO/AIO/AIO.bs
> > PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e"
> > "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
> > t/00_load.......ok
> > t/01_stat.......ok
> > t/02_read.......ok
> > t/03_errors.....ok
> > t/04_fork.......ok
> > t/05_readdir....ok
> > All tests successful.
> > Files=6, Tests=44, 0 wallclock secs ( 0.24 cusr + 0.09 csys = 0.33CPU)
> >
> > I hope you find this usefull :)
> >
> > Arnoud
> >
> > On 1/29/06, Brad Fitzpatrick <brad at danga.com> wrote:
> > >
> > > Are you using Linux::AIO or IO::AIO? My guess is you installed
> Linux::AIO
> > > but the test suite didn't pass. Go download it,
> > >
> > >
> http://search.cpan.org/CPAN/authors/id/M/ML/MLEHMANN/Linux-AIO-1.8.tar.gz
> > >
> > > and then:
> > >
> > > $ perl Makefile.PL
> > > $ make test
> > >
> > > What does it say?
> > >
> > > If it fails, could you also try IO::AIO and let me know what that test
> > > suite says. I may have to change the default preference to use
> IO::AIO
> > > instead of Linux::AIO.
> > >
> > >
> http://search.cpan.org/CPAN/authors/id/M/ML/MLEHMANN/IO-AIO-1.71.tar.gz
> > >
> > > (Anybody reading this that wants to test both, I'm interested in
> hearing
> > > your results....)
> > >
> > > - Brad
> > >
> > >
> > > On Sun, 29 Jan 2006, Arnoud Vermeer wrote:
> > >
> > > > I fixed the problem by manually creating the dir dev[456] in
> > > /var/mogdata.
> > > > It seems like there is a problem with the mogstored demon (I think?)
> > > > http://lists.danga.com/pipermail/mogilefs/2006-January/000244.htmlis
> > > having
> > > > the same problem as I am.
> > > >
> > > > I hope this problem can get fixed.
> > > >
> > > > Thanks,
> > > >
> > > > Arnoud Vermeer
> > > >
> > > > On 1/29/06, Arnoud Vermeer <a.vermeer at freshway.biz> wrote:
> > > > >
> > > > > Thanks for your replies.
> > > > >
> > > > > Regarding Joshua:
> > > > > mfs1:# df -l
> > > > > Filesystem 1K-blocks Used Available Use% Mounted on
> > > > > /dev/hda1 37776652 555432 35302260 2% /
> > > > > tmpfs 124472 0 124472 0% /dev/shm
> > > > >
> > > > > mfs2:# df -l
> > > > > Filesystem 1K-blocks Used Available Use% Mounted on
> > > > > /dev/hda1 37136300 501176 34748696 2% /
> > > > > tmpfs 225764 0 225764 0% /dev/shm
> > > > >
> > > > > mfs3:# df -l
> > > > > Filesystem 1K-blocks Used Available Use% Mounted on
> > > > > /dev/hda1 37744520 510772 35316392 2% /
> > > > > tmpfs 128520 0 128520 0% /dev/shm
> > > > >
> > > > > I didn't map any device to the devicenames in mogilefs, should I ?
> and
> > > > > how?
> > > > >
> > > > > Regarding brad:
> > > > >
> > > > > telnet localhost 7500
> > > > > Trying 127.0.0.1...
> > > > > Connected to localhost.localdomain.
> > > > > Escape character is '^]'.
> > > > > GET http://81.171.121.30:7500/dev4/usage
> > > > >
> > > > > doesn't return anything..
> > > > >
> > > > > This is my config:
> > > > > cat /etc/mogilefs/mogstored.conf
> > > > > httplisten = 0.0.0.0:7500
> > > > > docroot = /var/mogdata
> > > > >
> > > > > --
> > > > >
> > > > > cat /etc/mogilefs/mogilefsd.conf
> > > > > daemonize = 1
> > > > > db_dsn = DBI:mysql:mogilefs:host=111.111.111.43
> > > > > db_user = mogilefs
> > > > > db_pass = PASSWD
> > > > > conf_port = 7001
> > > > > listener_jobs = 10
> > > > > delete_jobs = 1
> > > > > replicate_jobs = 5
> > > > > mog_root = /mnt/mogilefs
> > > > > reaper_jobs = 1
> > > > > # debug = 3
> > > > >
> > > > > any suggestions?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Arnoud
> > > > >
> > > > > On 1/29/06, Brad Fitzpatrick <brad at danga.com> wrote:
> > > > > >
> > > > > > You have some port numbers wrong somewhere... the tracker
> doesn't
> > > speak
> > > > > > HTTP.
> > > > > >
> > > > > > - Brad
> > > > > >
> > > > > >
> > > > > > On Sat, 28 Jan 2006, Arnoud Vermeer wrote:
> > > > > >
> > > > > > > Hi guys,
> > > > > > >
> > > > > > > I'm trying to setup a small test cluster of MogileFS, but I'm
> > > running
> > > > > > into a
> > > > > > > problem.
> > > > > > > I've got 3 desktop PC's in the datacenter running Debian Sarge
> > > > > > (installed
> > > > > > > via netinstall)
> > > > > > > 2 x 1200 Mhz intel with a single PATA 40 gb HD
> > > > > > > and a AMD 2400+ also with a PATA 40 gb HD.
> > > > > > >
> > > > > > > I've installed mogileFS just fine, first the .tar.gz release
> from
> > > the
> > > > > > site,
> > > > > > > but that one was running into problems with adding disks.
> > > > > > > So I grabbed the latest CVS (28-jan-2006), but this one seems
> to
> > > run
> > > > > > into
> > > > > > > the same problems.
> > > > > > >
> > > > > > > mfs1 (1200Mhz+40GB HD) runs Perlbal, mogilefsd and mogstored
> > > > > > > mfs2 (2400Mhz+40GB HD) runs Perlbal, mogilefsd and mogstored
> > > > > > > mfs3 (1200Mhz+40GB HD) just runs mogstored
> > > > > > >
> > > > > > > I've added the hosts and disks via the mogadm commandline, but
> > > when I
> > > > > > run a:
> > > > > > > mogadm -c /etc/mogilefs/mogadm.conf check
> > > > > > >
> > > > > > > I get the following result:
> > > > > > >
> > > > > > > mogadm -c /etc/mogilefs/mogadm.conf check (IP's and domain
> > > removed..)
> > > > > > > Checking trackers...
> > > > > > > 111.111.111.30:7001 ... OK
> > > > > > > 111.111.111.31:7001 ... OK
> > > > > > >
> > > > > > > Checking hosts...
> > > > > > > [ 1] mfs1.domain ... OK
> > > > > > > [ 2] mfs2.domain ... OK
> > > > > > > [ 3] mfs3.domain ... OK
> > > > > > >
> > > > > > > Checking devices...
> > > > > > > host device size(G) used(G) free(G)
> > > use%
> > > > > > > ---- --------------- ---------- ---------- ----------
> > > ------
> > > > > > > [ 1] dev4 REQUEST FAILURE
> > > > > > > [ 2] dev3 REQUEST FAILURE
> > > > > > > [ 3] dev5 REQUEST FAILURE
> > > > > > > Illegal division by zero at /usr/bin/mogadm line 165,
> > > > > > > <Sock_81.171.121.31:7001> line 2.
> > > > > > >
> > > > > > > So I looked into the source and found the command at line 165,
> and
> > > > > > gave this
> > > > > > > via telnet onto the the tracker:
> > > > > > >
> > > > > > > telnet localhost 7001
> > > > > > > Trying 127.0.0.1...
> > > > > > > Connected to localhost.localdomain.
> > > > > > > Escape character is '^]'.
> > > > > > > GET http://81.171.121.30:7500/dev4/usage
> > > > > > > ERR unknown_command Unknown+server+command
> > > > > > >
> > > > > > > So now I'm stumped.
> > > > > > >
> > > > > > > Does anybody have a clue?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Arnoud Vermeer
> > > > > > > FreshWay Innovations
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/mogilefs/attachments/20060129/0a7688bb/attachment.htm
More information about the mogilefs
mailing list