Pluggable dictionaries

Gavin Carr gavin at openfusion.com.au
Fri Jan 15 12:51:56 UTC 2010


In merging Kostas' lowmem patches I realised that another place we're
using a lot of ram in brackup is the DigestCache and InventoryDatabase
handling. The storage backend for both (Dict::SQLite) is optimised for 
speed at the expense of memory - it just slurps the entire database 
into ram and then uses it from there.

So I've committed a change to make the dictionary type selectable for
both the digest db (SOURCE 'digestdb_type') and the inventory database
(TARGET 'inventorydb_type'), and added a Dict::SQLite2 module that
uses conventional db queries/iterators instead of loading the db into 
ram. Which is slower, but uses much less memory, so you can choose to 
trade speed for memory if you want to.

This also allows other kinds of dictionary implementations to be 
written. I'm benchmarking DBM and Redis implementations at the moment, 
and will commit them as well if the performance looks interesting.

Cheers,
Gavin



More information about the brackup mailing list