update on the low-memory patch

[Gavin Carr]

Hi Kostas,

On Tue, Dec 29, 2009 at 09:00:48PM +0200, Kostas Chatzikokolakis wrote:
> I've been using the latest version of the low-memory patch in all
> production systems I manage as well as for my personal backup for more
> than a month. So far it works perfectly, using very low memory and
> finishing in all cases where I was previously getting out of memory
> errors. Also the issues I was having with files being concurrently
> updated are gone now. Restore has also been tested several times.
> 
> Since other people were also experiencing memory issues I think it's
> worth the effort to consider the patch for merge. More testing by other
> people is very welcome.

I've started to look at your lowmem patches here:

  http://codereview.appspot.com/135046/show

It's a pretty big changeset, and it seems to me that there are at least 3 
different sets of changes here:

- the lowmem changes, passing around filehandles instead of file content

- the concurrent changes patches, for better handling of changes to files 
  as they're being backed up

- test changes, primarily the inclusion of check_inventory_db

Are these more interrelated than I'm seeing, or could they be split up for
easier review?

I did a little bit of testing today with this changeset, doing a medium
sized backup (3GB) to a local sftp target. RAM usage was great - 25% better 
than trunk - but it was also a fair bit slower (~30%). Haven't tried other
targets yet though, or looked at all into the performance issue.

Cheers,
Gavin