o_append with Perlbal::AIO?

[Brad Fitzpatrick]

Sorry, I was confusing lack of o_append in kernel with something else I
was reading recently[1].  My bad... o_append is handled specially.

So maybe it's fine?  Just do your aio_open with the flags including
o_append, and then aio_writes to that fd later should be appending... what
didn't work?


[1] Thread "sys_write() racy for multi-threaded append?" in:
    http://www.ussg.iu.edu/hypermail/linux/kernel/0703.1/index.html

On Tue, 20 Mar 2007, EKC wrote:

> Thanks for the response.
>
> Perhaps I don't understand o_append properly. I thought atomic
> o_append  was part of the POSIX standard and supported by 2.6? Am I
> mistaken?
>
> Since o_append is atomic, why do I need locking support amongst the
> AIO worker threads? If I do not care about the order of writes to the
> file, and if I only care about maintaining file integrity, what is the
> downside to multiple AIO worker threads doing concurrent o_append
> writes to the same file? How is that different from multiple
> independent processes doing concurrent o_append writes to the same
> file?
>
> My intention was to use atomic o_append, atomic rename, and mmap to
> write a cursory application-specific database. Is o_append support
> absent from the kernel or is there some issue with epoll?
>
> Thanks again
>
> On 3/19/07, Brad Fitzpatrick <brad at danga.com> wrote:
> > It's not going to happen:  the kernel doesn't support it, and there's no
> > locking between all the IO::AIO or Linux::AIO worker threads.
> >
> > You need to serialize the writes yourself (which is easy, since Perlbal's
> > a single-threaded event loop!), and just make sure you only have one
> > aio_write() outstanding at a time.  We do this already, for instance, in
> > the buffered upload writing to /tmp/.  When the one outstanding aio_write
> > completes, we see if we have another to do, and kick the next one off.
> > (which might already be merged from several writes that came in while the
> > real write was outstanding)
> >
> >
> > On Mon, 19 Mar 2007, EKC wrote:
> >
> > > Howdy,
> > >
> > > For the life of me, I can't seem to figure out how to do AIO o_append
> > > in Perlbal. I'm using Perlbal::AIO, which is using Linux::AIO from
> > > CPAN.
> > >
> > > What I would like to do, is to use o_append to log the activity of a
> > > custom Perlbal hook that I am developing. However, the only file
> > > writing operations supported by Linux::AIO and IO::AIO seem to be
> > > sendfile and aio_write, neither of which appends bytes atomically to
> > > the end of a file.
> > >
> > > I have multiple concurrent processes appending to the same file, and I
> > > would like to use AIO and o_append to be able to atomically AND
> > > asynchronously append to the file from within Perlbal.
> > >
> > > Is there any way to use o_append with Perlbal::AIO (or IO::AIO, or
> > > Linux::AIO) that I'm overlooking? Is there a better way of getting
> > > asynchrounous atomic o_append support into Perlbal?
> > >
> > > Thanks in advance!
> > >
> > > BTW, Perlbal rocks!
> > >
> > >
> >
> >
>
>