Files to delete
Brad Fitzpatrick
brad at danga.com
Wed May 16 21:52:27 UTC 2007
Adam,
That was fixed in 2.16.
On Wed, 16 May 2007, Adam Rosien wrote:
> Found lots of messages in /var/log/messages:
>
> May 16 11:47:32 arosien-dtl mogilefsd[30396]: crash log: Invalid devid
> at /usr/lib/perl5/site_perl/5.8.5/MogileFS/Worker/Delete.pm line 172
>
> .. Adam
>
> On 5/16/07, Adam Rosien <adam at rosien.net> wrote:
> > I get:
> >
> > [arosien at arosien-dtl ~]$ telnet localhost 16001
> > Trying 127.0.0.1...
> > Connected to localhost.localdomain (127.0.0.1).
> > Escape character is '^]'.
> > !watch
> > Added you to watcher list.
> > .
> > :: Child 30030 (delete) died: 256 (UNEXPECTED)
> > :: Job delete has only 0, wants 1, making 1.
> >
> > And lots more of the child died messages. The file_to_delete and
> > _file_to_delete_later tables have lots of rows.
> >
> > One problem I can imagine is that some of the storage nodes be running
> > older mogstored and some protocol changed between the tracker and
> > mogstored?
> >
> > .. Adam
> >
> > On 5/12/07, Oezcan Acar <acar at unitedinter.net> wrote:
> > > mysql> select * from file_to_delete;
> > > Empty set (0,00 sec)
> > >
> > > mysql> select * from file_to_delete_later;
> > > Empty set (0,00 sec)
> > >
> > >
> > > bizimalem at tracker1:~> telnet localhost 6001
> > > Trying 127.0.0.1...
> > > Connected to localhost.
> > > Escape character is '^]'.
> > > !watch
> > > Added you to watcher list.
> > > .
> > >
> > > .
> > >
> > >
> > > -------- Original-Nachricht --------
> > > Datum: Sat, 12 May 2007 20:35:06 +0000 (UTC)
> > > Von: Brad Fitzpatrick <brad at danga.com>
> > > An: Oezcan Acar <acar at unitedinter.net>
> > > CC: mogilefs at lists.danga.com
> > > Betreff: Re: Files to delete
> > >
> > > > What's in your file_to_delete and file_to_delete_later tables?
> > > >
> > > > If you telnet to mogielfsd, on port 7001, and type "!watch", do you see
> > > > any error messages about deleting?
> > > >
> > > > Deletes should happen in seconds, not 6+ hours.
> > > >
> > > >
> > > > On Sat, 12 May 2007, Oezcan Acar wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I deleted the files 6 hours ago, but they are not removed from the
> > > > filesystem
> > > > >
> > > > >
> > > > > ps afx shows:
> > > > >
> > > > > 21144 ? S 5:07 /usr/bin/perl /usr/bin/mogilefsd -c
> > > > /etc/mogilefs/mogilefsd.conf --daemon
> > > > > 21145 ? S 3:22 \_ /usr/bin/mogilefsd [replicate]
> > > > > 21146 ? S 0:00 \_ /usr/bin/mogilefsd [delete]
> > > > > 21147 ? S 0:37 \_ /usr/bin/mogilefsd [queryworker]
> > > > > 21148 ? S 0:39 \_ /usr/bin/mogilefsd [queryworker]
> > > > > 21149 ? S 0:40 \_ /usr/bin/mogilefsd [queryworker]
> > > > > 21150 ? S 0:43 \_ /usr/bin/mogilefsd [queryworker]
> > > > > 21151 ? S 0:56 \_ /usr/bin/mogilefsd [queryworker]
> > > > > 21152 ? S 14:24 \_ /usr/bin/mogilefsd [monitor]
> > > > > 21153 ? S 0:54 \_ /usr/bin/mogilefsd [reaper]
> > > > > 21154 ? SN 0:49 \_ /usr/bin/mogilefsd [fsck]
> > > > >
> > > > >
> > > > > I am using the HEAD version and my mogilefsd.conf looks like:
> > > > >
> > > > >
> > > > > db_dsn DBI:mysql:mogilefs:192.168.1.30
> > > > > db_user bizimalem
> > > > > db_pass bizimalem
> > > > > conf_port 6001
> > > > > listener_jobs 5
> > > > >
> > > > >
> > > > > When will be the files removed from the underlying filesystem?
> > > > >
> > > > > thanks.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > -------- Original-Nachricht --------
> > > > > Datum: Sat, 12 May 2007 16:34:35 +0000 (UTC)
> > > > > Von: Brad Fitzpatrick <brad at danga.com>
> > > > > An: Oezcan Acar <acar at unitedinter.net>
> > > > > CC: mogilefs at lists.danga.com
> > > > > Betreff: Re: Files to delete
> > > > >
> > > > > > A delete operation just removes it from the namespace immediately.
> > > > The
> > > > > > actual deletions from mogstoreds (which may be down at the time of
> > > > your
> > > > > > delete!), happen asynchronously, sometime in the future... usually
> > > > within
> > > > > > a 10-15 seconds, but more if there were problems.
> > > > > >
> > > > > > See the file_to_delete table (immediately enqueued), or
> > > > > > file_to_delete_later (where it goes if there are problems while
> > > > deleting).
> > > > > >
> > > > > > Make sure you're running [delete] jobs on your mogilefsds...
> > > > > >
> > > > > > $ ps afx | grep delete
> > > > > >
> > > > > > You see a line like:
> > > > > >
> > > > > > \--- mogilefsd [delete]
> > > > > >
> > > > > > If so, that's what will delete it later.
> > > > > >
> > > > > > - Brad
> > > > > >
> > > > > >
> > > > > > On Sat, 12 May 2007, Oezcan Acar wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I added with mogtool new files to mogilefs and deleted them again
> > > > with
> > > > > > mogadmin.
> > > > > > >
> > > > > > >
> > > > > > > mogtool --trackers=192.168.1.2:6001 --domain=ref --class=class
> > > > delete
> > > > > > mykey
> > > > > > >
> > > > > > > After this op mogtool says "Deleted" but the files are not removed
> > > > from
> > > > > > devices! Is that a correct behaviour or are the files marked as to be
> > > > > > deleted later? How is the file deletion policy ?
> > > > > > >
> > > > > > > thanks.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Mit freundlichen Grüßen
> > > > > > >
> > > > > > > ___________________________
> > > > > > >
> > > > > > > Özcan Acar, Dipl. Inf(FH)
> > > > > > > Vorstand
> > > > > > > Smart Web Business AG
> > > > > > >
> > > > > > > Im Niederfeld 8
> > > > > > > 65462 Gustavsburg
> > > > > > >
> > > > > > > +49 171 57 33 66 5
> > > > > > > acar at unitedinter.net
> > > > > > >
> > > > > > >
> > > > >
> > > > > --
> > > > > Mit freundlichen Grüßen
> > > > >
> > > > > ___________________________
> > > > >
> > > > > Özcan Acar, Dipl. Inf(FH)
> > > > > Vorstand
> > > > > Smart Web Business AG
> > > > >
> > > > > Im Niederfeld 8
> > > > > 65462 Gustavsburg
> > > > >
> > > > > +49 171 57 33 66 5
> > > > > acar at unitedinter.net
> > > > >
> > > > >
> > >
> > > --
> > > Mit freundlichen Grüßen
> > >
> > > ___________________________
> > >
> > > Özcan Acar, Dipl. Inf(FH)
> > > Vorstand
> > > Smart Web Business AG
> > >
> > > Im Niederfeld 8
> > > 65462 Gustavsburg
> > >
> > > +49 171 57 33 66 5
> > > acar at unitedinter.net
> > >
> > >
> >
>
>
More information about the mogilefs
mailing list