Pluggable dictionaries
Gavin Carr
gavin at openfusion.com.au
Fri Jan 15 12:51:56 UTC 2010
In merging Kostas' lowmem patches I realised that another place we're
using a lot of ram in brackup is the DigestCache and InventoryDatabase
handling. The storage backend for both (Dict::SQLite) is optimised for
speed at the expense of memory - it just slurps the entire database
into ram and then uses it from there.
So I've committed a change to make the dictionary type selectable for
both the digest db (SOURCE 'digestdb_type') and the inventory database
(TARGET 'inventorydb_type'), and added a Dict::SQLite2 module that
uses conventional db queries/iterators instead of loading the db into
ram. Which is slower, but uses much less memory, so you can choose to
trade speed for memory if you want to.
This also allows other kinds of dictionary implementations to be
written. I'm benchmarking DBM and Redis implementations at the moment,
and will commit them as well if the performance looks interesting.
Cheers,
Gavin
More information about the brackup
mailing list