Lucene and memcached

[Kevin A. Burton]

Gregory Block wrote:

>
> If you can build a way to create a hash representing a query, you can 
> cache the full results of the query.
>
> However: You're probably much, much better off spending some time 
> optimising the way you query in the first place. Lucene has a complex 
> relationship between search performance, searchers, and some of the 
> objects which, while you *can* instantiate copies each time, having 
> good performance requires that you keep around.
>
> Assuming, of course, that you're probably doing what everyone does 
> first time around: creating index Readers and Searchers on each 
> request, and not using any of the cacheable representations of those 
> objects.
>
> That's your first port of call.
>
Actually Lucene filters are something that can benefit from caching.  
For example lets say you wanted to let users search JUST on the last 30 
days worth of content.  You could run the filter ONCE and then cache it 
with the Term clause used to run the query.

Then you could just use the same filter again for every user until you 
have to optimize() the index again.  As long as the document numbers 
stay they same you don't have much more to do...

Kevin

-- 

Use Rojo (RSS/Atom aggregator).  Visit http://rojo.com. Ask me for an 
invite!  Also see irc.freenode.net #rojo if you want to chat.

Rojo is Hiring! - http://www.rojonetworks.com/JobsAtRojo.html

If you're interested in RSS, Weblogs, Social Networking, etc... then you 
should work for Rojo!  If you recommend someone and we hire them you'll 
get a free iPod!
    
Kevin A. Burton, Location - San Francisco, CA
       AIM/YIM - sfburtonator,  Web - http://peerfear.org/
GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412