Overview
Allow users to add a Lucene index on a region that already exists and contains data. We also want to simplify the process to modify a Lucene index on a region. To modify an index, the user will need to first destroy the existing Lucene index and then add a new index, without having to destroy and recreate the data region, which is required today.
Goals:
- Lucene Index can be created before or after a data region has been created
- Support an active cluster (puts in flight) when adding Lucene index
Index both existing data in region as well as new data events
Be able to add Lucene index from gfsh and have it stored in cluster config
Can handle HA events (members dying, new members added, rebalancing)
Need a public Java API to do distributed creation of Lucene Index
Queries on indexes that are in the middle of being initialized should throw an exception
Approach
1. User initiates a create lucene index command
A function is sent in parallel that does the following:
- Create AEQ
- Create index region
- Add AEQ and listener to region
- Return xml
- Send xml to locator to be written into cluster configuration if applicable
- Another function is sent in parallel to:
- Set indexRepositories()
- computeRepo() will look for COMPLETE file; if doesn’t exist
- Iterate and index existing region data
- Add COMPLETE file to fileAndChunkRegion
- Query:
- Check for COMPLETE file
- If COMPLETE file not there, start async task to computeRepo(), and throw an exception
- Check for COMPLETE file
API Change
- A new Java API to create the Lucene Index in a distributed manor