Proposal Clusterable Search Via Solr

Abstract

Roller uses Lucene for search and Lucene stores its search index on disk. So, if you have multiple Roller instances running you can either 1) have both Lucene instances write to the same disk file which will fail 2) have two inconsistent search indexes which will be extremely irritating or 3) turn off Roller's build-in search and use some external spider.

For those who are not happy with those choices, I offer this proposal to use a) Apache Solr and b) some improvements to Roller's plugin infrastructure to enable a clusterable.

Design

The basic idea is to embed the Apache Solr in Roller. Solr maintains a search index and can perform queries against that index and all of this can be done via web services. To be continued...

• embed Solr search engine in Roller, and expose it's web services interface
  • allow it to run on one host of the system
  • all hosts will call the search service via web services
• introduce WeblogListener and supporting infrastructure
  • entryAdded
  • entryUpdated
  • entryRemoved
  • spamCommentDetected
  • commentAdded
  • commentUpdated
  • commentRemoved
• Create new SolrSearchManager implementation that calls Solr Web Services to manage the index, perform searches, etc.
• Create a SearchWeblogListener implementation that indexes, re-indexes and de-indexes as needed by calling the SearchManager interface
• Replace calls in Struts actions to search manager for indexing, re-indexing, etc. They are no longer needed now that SearchWeblogListener is in place.

And we're done. Folks who want clustering can use the new SolrSearchManager, folks who don't can continue to use LucenseSearchManager.