Files to delete

[Adam Rosien]

Found lots of messages in /var/log/messages:

May 16 11:47:32 arosien-dtl mogilefsd[30396]: crash log: Invalid devid
at /usr/lib/perl5/site_perl/5.8.5/MogileFS/Worker/Delete.pm line 172

.. Adam

On 5/16/07, Adam Rosien <adam at rosien.net> wrote:
> I get:
>
> [arosien at arosien-dtl ~]$ telnet localhost 16001
> Trying 127.0.0.1...
> Connected to localhost.localdomain (127.0.0.1).
> Escape character is '^]'.
> !watch
> Added you to watcher list.
> .
> :: Child 30030 (delete) died: 256 (UNEXPECTED)
> :: Job delete has only 0, wants 1, making 1.
>
> And lots more of the child died messages.  The file_to_delete and
> _file_to_delete_later tables have lots of rows.
>
> One problem I can imagine is that some of the storage nodes be running
> older mogstored and some protocol changed between the tracker and
> mogstored?
>
> .. Adam
>
> On 5/12/07, Oezcan Acar <acar at unitedinter.net> wrote:
> > mysql> select * from file_to_delete;
> > Empty set (0,00 sec)
> >
> > mysql> select * from file_to_delete_later;
> > Empty set (0,00 sec)
> >
> >
> > bizimalem at tracker1:~> telnet localhost 6001
> > Trying 127.0.0.1...
> > Connected to localhost.
> > Escape character is '^]'.
> > !watch
> > Added you to watcher list.
> > .
> >
> > .
> >
> >
> > -------- Original-Nachricht --------
> > Datum: Sat, 12 May 2007 20:35:06 +0000 (UTC)
> > Von: Brad Fitzpatrick <brad at danga.com>
> > An: Oezcan Acar <acar at unitedinter.net>
> > CC: mogilefs at lists.danga.com
> > Betreff: Re: Files to delete
> >
> > > What's in your file_to_delete and file_to_delete_later tables?
> > >
> > > If you telnet to mogielfsd, on port 7001, and type "!watch", do you see
> > > any error messages about deleting?
> > >
> > > Deletes should happen in seconds, not 6+ hours.
> > >
> > >
> > > On Sat, 12 May 2007, Oezcan Acar wrote:
> > >
> > > > Hi,
> > > >
> > > > I deleted the files 6 hours ago, but they are not removed from the
> > > filesystem
> > > >
> > > >
> > > > ps afx shows:
> > > >
> > > > 21144 ?        S      5:07 /usr/bin/perl /usr/bin/mogilefsd -c
> > > /etc/mogilefs/mogilefsd.conf --daemon
> > > > 21145 ?        S      3:22  \_ /usr/bin/mogilefsd [replicate]
> > > > 21146 ?        S      0:00  \_ /usr/bin/mogilefsd [delete]
> > > > 21147 ?        S      0:37  \_ /usr/bin/mogilefsd [queryworker]
> > > > 21148 ?        S      0:39  \_ /usr/bin/mogilefsd [queryworker]
> > > > 21149 ?        S      0:40  \_ /usr/bin/mogilefsd [queryworker]
> > > > 21150 ?        S      0:43  \_ /usr/bin/mogilefsd [queryworker]
> > > > 21151 ?        S      0:56  \_ /usr/bin/mogilefsd [queryworker]
> > > > 21152 ?        S     14:24  \_ /usr/bin/mogilefsd [monitor]
> > > > 21153 ?        S      0:54  \_ /usr/bin/mogilefsd [reaper]
> > > > 21154 ?        SN     0:49  \_ /usr/bin/mogilefsd [fsck]
> > > >
> > > >
> > > > I am using the HEAD version and my mogilefsd.conf looks like:
> > > >
> > > >
> > > > db_dsn DBI:mysql:mogilefs:192.168.1.30
> > > > db_user bizimalem
> > > > db_pass bizimalem
> > > > conf_port 6001
> > > > listener_jobs 5
> > > >
> > > >
> > > > When will be the files removed from the underlying filesystem?
> > > >
> > > > thanks.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > -------- Original-Nachricht --------
> > > > Datum: Sat, 12 May 2007 16:34:35 +0000 (UTC)
> > > > Von: Brad Fitzpatrick <brad at danga.com>
> > > > An: Oezcan Acar <acar at unitedinter.net>
> > > > CC: mogilefs at lists.danga.com
> > > > Betreff: Re: Files to delete
> > > >
> > > > > A delete operation just removes it from the namespace immediately.
> > > The
> > > > > actual deletions from mogstoreds (which may be down at the time of
> > > your
> > > > > delete!), happen asynchronously, sometime in the future... usually
> > > within
> > > > > a 10-15 seconds, but more if there were problems.
> > > > >
> > > > > See the file_to_delete table (immediately enqueued), or
> > > > > file_to_delete_later (where it goes if there are problems while
> > > deleting).
> > > > >
> > > > > Make sure you're running [delete] jobs on your mogilefsds...
> > > > >
> > > > >    $ ps afx | grep delete
> > > > >
> > > > > You see a line like:
> > > > >
> > > > >      \--- mogilefsd [delete]
> > > > >
> > > > > If so, that's what will delete it later.
> > > > >
> > > > > - Brad
> > > > >
> > > > >
> > > > > On Sat, 12 May 2007, Oezcan Acar wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I added with mogtool new files to mogilefs and deleted them again
> > > with
> > > > > mogadmin.
> > > > > >
> > > > > >
> > > > > > mogtool --trackers=192.168.1.2:6001 --domain=ref --class=class
> > > delete
> > > > > mykey
> > > > > >
> > > > > > After this op mogtool says "Deleted" but the files are not removed
> > > from
> > > > > devices! Is that a correct behaviour or are the files marked as to be
> > > > > deleted later? How is the file deletion policy ?
> > > > > >
> > > > > > thanks.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Mit freundlichen Grüßen
> > > > > >
> > > > > > ___________________________
> > > > > >
> > > > > > Özcan Acar, Dipl. Inf(FH)
> > > > > > Vorstand
> > > > > > Smart Web Business AG
> > > > > >
> > > > > > Im Niederfeld 8
> > > > > > 65462 Gustavsburg
> > > > > >
> > > > > > +49 171 57 33 66 5
> > > > > > acar at unitedinter.net
> > > > > >
> > > > > >
> > > >
> > > > --
> > > > Mit freundlichen Grüßen
> > > >
> > > > ___________________________
> > > >
> > > > Özcan Acar, Dipl. Inf(FH)
> > > > Vorstand
> > > > Smart Web Business AG
> > > >
> > > > Im Niederfeld 8
> > > > 65462 Gustavsburg
> > > >
> > > > +49 171 57 33 66 5
> > > > acar at unitedinter.net
> > > >
> > > >
> >
> > --
> > Mit freundlichen Grüßen
> >
> > ___________________________
> >
> > Özcan Acar, Dipl. Inf(FH)
> > Vorstand
> > Smart Web Business AG
> >
> > Im Niederfeld 8
> > 65462 Gustavsburg
> >
> > +49 171 57 33 66 5
> > acar at unitedinter.net
> >
> >
>