<div dir="ltr"><div class="gmail_quote">On Tue, Jul 15, 2008 at 2:51 PM, Michael Hanselmann <public@hansmi.ch> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi Brad<br>
<br><div class="Ih2E3d">> Uh, you do this unconditionally? Not dependent on the size of the file!?<br>
> How am I going to backup a huge file on a low-memory box?<br>
<br>
</div>Yes and no. The chunker is responsible to return sensible values for<br>
$len (see the other two patches). However, I just realized that this can<br>
indeed be problematic for large files because everything is kept in<br>
memory until the file is done.<br>
<br>
One solution would be to basically stream everything, but that's not<br>
such a simple change.<br>
<br>
Do you have an idea?</blockquote><div><br></div><div>Did you even measure that this patch had any positive effect, even for small files? Because if you just read the small file, it should still be in the kernel's page cache and reading it should incur no extra I/O.</div>
<div><br></div><div>So as far as I see, you just made the big file case worse for no gain on small files.</div><div><br></div><div>But if you did measure a notable gain, 1) post numbers and 2) only seed the positionchunk with its dataref if the file is 'small', else make the chunkref accessor read the file on-demand, like it used to. Best of both worlds.</div>
<div><br></div><div>- Brad</div><div><br></div></div></div>