You are viewing an old version of this page. View the current version.

Compare with Current View Page History

Version 1 Next »

Overview

Allow users to add a Lucene index on a region that already exists and contains data. We also want to simplify the process to modify a Lucene index on a region. To modify an index, the user will need to first destroy the existing Lucene index and then add a new index, without having to destroy and recreate the data region, which is required today.

Goals:

  1. Lucene Index can be created before or after a data region has been created
  2. Support an active cluster (puts in flight) when adding Lucene index

  3. Index both existing data in region as well as new data events

  4. Be able to add Lucene index from gfsh and have it stored in cluster config

  5. Can handle HA events (members dying, new members added, rebalancing)

  6. Need a public Java API to do distributed creation of Lucene Index

  7. Queries on indexes that are in the middle of being initialized should throw an exception


    Approach

  1.   1. User initiates a create lucene index command

    1. A function is sent in parallel that does the following:

      1. Create AEQ
      2. Create index region
      3. Add AEQ and listener to region
      4. Return xml
    2. Send xml to locator to be written into cluster configuration if applicable
    3. Another function is sent in parallel to:
      1. Set indexRepositories()
      2. computeRepo() will look for COMPLETE file; if doesn’t exist
        1. Iterate and index existing region data
        2. Add COMPLETE file to fileAndChunkRegion
  2. Query:
    1. Check for COMPLETE file
    2. If COMPLETE file not there, start async task to computeRepo(), and throw an exception

API Change

  • A new Java API to create the Lucene Index in a distributed manor

 

  • No labels