[PATCH 3/3] Open files only once

[Brad Fitzpatrick]

On Tue, Jul 15, 2008 at 2:51 PM, Michael Hanselmann <public at hansmi.ch>
wrote:

> Hi Brad
>
> > Uh, you do this unconditionally?  Not dependent on the size of the file!?
> > How am I going to backup a huge file on a low-memory box?
>
> Yes and no. The chunker is responsible to return sensible values for
> $len (see the other two patches). However, I just realized that this can
> indeed be problematic for large files because everything is kept in
> memory until the file is done.
>
> One solution would be to basically stream everything, but that's not
> such a simple change.
>
> Do you have an idea?


Did you even measure that this patch had any positive effect, even for small
files?  Because if you just read the small file, it should still be in the
kernel's page cache and reading it should incur no extra I/O.

So as far as I see, you just made the big file case worse for no gain on
small files.

But if you did measure a notable gain, 1) post numbers and 2) only seed the
positionchunk with its dataref if the file is 'small', else make the
chunkref accessor read the file on-demand, like it used to.  Best of both
worlds.

- Brad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.danga.com/pipermail/brackup/attachments/20080716/56722c4c/attachment.html