Fwd: buggy tracker behavior?

[Drop Box]

I forgot to cc the list.  Sorry for the duplicate, Junior.

---------- Forwarded message ----------
From: Drop Box <publicdropbox at gmail.com>
Date: Feb 16, 2006 1:13 PM
Subject: Re: buggy tracker behavior?
To: Mark Smith <junior at danga.com>

Junior,

That was helpful, thanks.  The idea that the tracker will only verify one
path (in the absence of noverify) but return two paths seems
unintuitive/misleading.  After all, the returned array is not an array full
of verified paths.  At the same time, I can see how you wouldn't want to
take the time to verify two paths.  I don't have a thought-out proposal for
how I think it should be different, though, so I won't press the issue.

I'm using the (mostly complete but then apparently dropped) PHP client
written by the wikipedia folks, and the problem was that the first element
in the (associative) array of paths was path2 (I assume because path2 is
always returned first in the by the tracker, despite the fact that it hasn't
been verified), and it just did a "foreach @paths" sort of thing.
Alphabetizing the array puts path1 first and made things a little better.
There's still a lot of slowness associated with having to verify the path
about half the time (with 1 of 2 hosts down) on pages with 30+ pieces of
content.  Using "mogilefsd --node_timeout 1" didn't seem to speed things up
as I had expected, either, but my expectations could very well be off too.

If I recall correctly, there was talk of automatically marking systems or
devices as down in the database.  Has there been any progress on that?  If
not, is that something that should be done under job_monitor() in mogilefsd?


Thanks.

On 2/13/06, Mark Smith <junior at danga.com > wrote:
>
> The 'verify' option (or rather, the lack of the noverify option) only
> looks for one valid path.  Additionally, it randomizes based on the
> weights
> of the devices.  So you end up with one of two scenarios: it tries the
> good
> device first, or it tries the bad one first.
>
> If it tries the good one, it says "hey, got a good device!" and it returns
> both paths.  (No need to check the other one -- we've already verified one
> of them!)  If it hits the bad one, then it has to throw it out, and then
> return the other one.
>
> If you notice from your output, the ONLY time you see both is when the
> "good" device is listed as "path1" -- the first device it checked, and the
>
> one that came out okay.
>
> So no, not a bug.  Hope that makes sense.  :-)
>
>
> -jr
>
> On Thu, Feb 09, 2006, Drop Box wrote:
> > Hi,
> >
> > I'm testing out MogileFS and am getting some results from the tracker
> that I
> > didn't expect.  I'm wondering whether it's a bug or a misunderstanding
> on my
> > part.  I have a file saved to two devices on two different systems.  I
> > unplugged one of the systems/devices (dev1 on .22) from the network and
> > noticed that the tracker will sometimes respond to GET_PATHS with 2
> paths,
> > for some reason:
> >
> > $ telnet localhost 7001
> > Trying 127.0.0.1...
> > Connected to localhost ( 127.0.0.1).
> > Escape character is '^]'.
> > GET_PATHS domain=test&key=thisismykey
> > OK path2=http://192.168.0.22:7500/dev1/0/000/000/0000000054.fid&path1=http://192.168.0.23:7500/dev2/0/000/000/0000000054.fid&paths=2
> > GET_PATHS domain=test&key=thisismykey
> > OK path1= http://192.168.0.23:7500/dev2/0/000/000/0000000054.fid&paths=1
> > GET_PATHS domain=test&key=thisismykey
> > OK path2=http://192.168.0.22:7500/dev1/0/000/000/0000000054.fid&path1=http://192.168.0.23:7500/dev2/0/000/000/0000000054.fid&paths=2
> > GET_PATHS domain=test&key=thisismykey
> > OK path1= http://192.168.0.23:7500/dev2/0/000/000/0000000054.fid&paths=1
> > GET_PATHS domain=test&key=thisismykey
> > OK path2=http://192.168.0.22:7500/dev1/0/000/000/0000000054.fid&path1=http://192.168.0.23:7500/dev2/0/000/000/0000000054.fid&paths=2
> > GET_PATHS domain=test&key=thisismykey
> > OK path1= http://192.168.0.23:7500/dev2/0/000/000/0000000054.fid&paths=1
> > GET_PATHS domain=test&key=thisismykey
> > OK path1= http://192.168.0.23:7500/dev2/0/000/000/0000000054.fid&paths=1
>
> > GET_PATHS domain=test&key=thisismykey
> > OK path1= http://192.168.0.23:7500/dev2/0/000/000/0000000054.fid&paths=1
> > GET_PATHS domain=test&key=thisismykey
> > OK path2=http://192.168.0.22:7500/dev1/0/000/000/0000000054.fid&path1=http://192.168.0.23:7500/dev2/0/000/000/0000000054.fid&paths=2
>
> >
> >
> > I would have guessed that GET_PATHS would only return the 1 available
> path,
> > given that I'm not setting noverify.  Whether or not that's how it's
> > supposed to work (could someone verify whether my guess is correct,
> > though?), my knee-jerk reaction to the sporadic behavior is to think
> there's
> > something wrong.  Thoughts?
> >
> > I'm using the latest revision from CVS, btw.
> >
> > Thanks.
>
>
> --
> Junior (aka Mark Smith)
> junior at danga.com
>
> Software Engineer
> Six Apart / Danga Interactive
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/mogilefs/attachments/20060216/45d3b8a4/attachment.htm