Kostas Chatzikokolakis' lowmem patches merged

Gavin Carr gavin at openfusion.com.au
Tue Jan 12 13:59:41 UTC 2010


I've merged Kostas Chatzikokolakis' lowmem patches from here:

  http://codereview.appspot.com/135046/show

into trunk, as they give a significant drop in memory usage without 
any significant impact on performance. The basic idea is to pass around 
content via filehandles instead of scalars where that makes sense.

I've tried to split the lowmem patches there out from the other stuff
in that changeset (concurrent changes protection and additional tests),
in order to keep the scope of the changes down, but they're still 
pretty significant. (Kostas, it would be great if you had time to review 
my changes to make sure I haven't done something stupid when I've pulled
your changeset apart). I've also changed metafile storage to work the
same way as in Kostas' patches.

All tests are passing, and I've been running these changes in production
for a few days, but it would be great if some other people had time to 
test trunk against various targets (Amazon is only lightly tested so far,
and CloudFiles not at all) and could report their results. FWIW, I'm 
seeing memory usage drops of 25-30% with Filesystem, Ftp and Sftp targets.

These changes add one additional dependency, on IO::InnerFile, from the
IO::stringy distribution.

Cheers,
Gavin



More information about the brackup mailing list