Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. Java API: CacheConfiguration.setQueryEntity.setIndexes(QueryIndex idxs... );
  2. Java API: CacheConfiguration.setIndexedTypes + annotation QuerySqlField.index;
  3. SQL API: Create index with SQL query.

For Java API (ways 1, 2) an index is described with (IgniteCache, CacheValue.class, fields, indexName).

For SQL API (way 3) an index is described with (Schema, Table, fields, indexName).Every index created with those API can be described with: CacheContext, cache value class / type, index name, fields 

Index Query API will support those different index descriptions. API provides:

  1. Public class IndexQuery for building index query:
    1. it inherits public class Query<R> with R = Cache.Entry<K, V>;
    2. it accepts index descriptions at a constructor;
    3. it accepts index conditions (joint with AND) at object methods.
  2. User provides index conditions that matches index fields and their order (ts desc, price asc) to leverage on an index structure.

...

Code Block
languagejava
// Creates an index query for an index, created with Java API.specifying value class or value type: 
// 1. Specify index description at constructor.
// 2. Index name is optional param, can try do best to find right index basing on specified Value.class in constructor and fields in conditionscriteria fields.
// 3. Index conditionscriteria (joint with AND operation) with methods. Order of fields in criteria doesn't matter

QueryCursor<Cache.Entry<Long, Good>> cursor = ignite.cache("GOOD").query(
	new IndexQuery<Long, Good>(Good.class, idxName?)  // idxName is optional.
		.criteriasetCriteria(gt("ts", lastMidnightTs), lt("price", 123.0))
);

// Create index with SQL query: "CREATE INDEX GOOD_TS_DESC_PRICE_ASC_IDX on GOOD_TBL (ts DESC, price ASC)"
// 1. Table name should be specified because it is possible to have the same index name for different tables (e.g., __key_PK).
// 2. Index name is optional too (do our best to find right index to run the query).
QueryCursor<Cache.Entry<Long, Good>> cursor = ignite.cache("GOOD"QueryCursor<Cache.Entry<Long, Good>> cursor = ignite.cache("GOOD").query(
	new IndexQuery<Long, Good>("GOOD_TBLTYPE", "GOOD_TS_DESC_PRICE_ASC_IDX"idxName?)  // idxName is optional.
		.criteriasetCriteria(gt("ts", lastMidnightTs), lt("price", 123.0))
);

...

Code Block
languagejava
// Public packages.

// IndexQuery extends
public IndexQuery<K, V> extends Query<Cache.Entry<K, V>> {

	private List<IndexCriteria> criteria = new ArrayList<>();

	// Index description.
	private @Nullable String idxName;
	private @Nullable String valClassvalType;

	public IndexQuery lt(String field, Object val) {
		criteria.add(IndexCriteriaBuilder.lt(field, val));

		return this;
	}

	// Other methods are:
	// eq, notEq, gt, gte, lt, lte, between, in, notIn, min, max, predicate
}

// Internal packages.

class IndexCriteriaBuilder {
	public static IndexCriteria lt(String field, Object val);
}

abstract class IndexCriteria extends Serializable {
	private final List<String> fields;
}

// min, max
class MinMaxIndexCriteria extends IndexCondition {
	private final boolean max;
}

// gt, gte, lt, lte, between
class RangeIndexCriteria extends IndexCriteria {
	private final @Nullable Object lower;
	private final @Nullable Object upper;

	private final boolean lowerInclusive;
	private final boolean upperInclusive;
}

// in, notIn, eq, notEq
class InIndexCriteria extends IndexCriteria {
	private final Object[] vals;

	// Flag for not-in condition.
	private final boolean inverse;
}

// predicate
class PredicateIndexCriteria extends IndexCriteria {
	private IgnitePredicate<?> predicate;
}

...

  1. Introduce new type of query - INDEX;
  2. Final query processing is performed within IndexQueryProcessor;
  3. Entrypoint for distributed index queries is the IndexQueryProcessor.queryDistributed method, it executes MapReduce querythat leverage on other cache queries (ScanQuery, TextQuery):
    1. the Map phase is node-local querying of index, it returns sorted data by definitionof index, it returns sorted data by definition;
    2. filtering with IndexQuery.setFilter is analogue of ScanQuery.setFilter;
    3. the Reduce phase is responsible for the Merge Sort of data from multiple nodes;
    4. for implementing it's suggested to extend the h2.twostep package - move basic functionality to the ignite-core module, then different query processors can extends them (h2, index query processor).
  4. Entrypoint for local query is the IndexQueryProcessor.queryLocal method. It accepts users IndexQueryand IndexingQueryFilter for filtering result cache entries (primary partition, MVCC, etc);The method predicate accepts a function, that should be deployed on other nodes with GridCacheDeploymentManager.


Code Block
languagejava
// Internal package.

public class IndexQueryProcessor extends GridProcessorAdapter {
	// Provides access to indexes.
	private final IndexProcessor processor;

	// Entrypoint for local query.
    public Iterator<Cache.Entry<?,?>> queryLocal(IndexQuery idxQuery, @Nullable IndexingQueryFilter filters) throws IgniteException {
		// 1. If user specified index name, then check a query:
		// - sort query fields in index key order;
		// - check that it's a valid index query: fields covers index keys (from left to right)
		// - fail otherwise.
		// 2. If user doesn't specified index name:
		// - get all indexes for specified cache and Value.class;
		// - find index by filtering by a query fields (index keys must be in the same order as specified query fields, try sort fields to match any index).
		// - validate index query (see validation steps from 1.)
		Index idx = index(idxQuery.desc());

		// 1. Parse index conditions.
		// 2. Validate index condition, index type.
		// 3. Maps index conditions to an index query methods.
		// 4. Perform index operations, get Cursor.
		GridCursor<IndexRow> cursor = query(idx, idxQry.conditions());

		// 1. Map IndexRow to Cache entry.
		// 2. Apply specified cache filters (primary partitions, MVCC versions, etc.)
		// 3. Wrap cursor to iterator and return.
		return map_and_filter(cursor, filters);
	}

	private GridCursor<IndexRow> query(Index idx, List<IndexCondition> conditions) {
		// eq(key) -> idx.findOne(key)
		// notEq(key) -> idx.find(null, null, current -> current != key)
		// gt(key) -> idx.find(key, null, current -> current != key)
		// gte(key) -> idx.find(key, null)
		// lt(key) -> idx.find(null, key, current -> current != key)
		// lte(key) -> idx.find(null, key)
		// between(left, right) -> idx.find(left, right), inclusive only
		// in(keys...) -> idx.findOne(keys[0]) && idx.findOne(keys[1]) ...
		// notIn(keys...) -> idx.find(null, null, current -> !keys.contains(current))
		// min() -> idx.findFirst()
		// max() -> idx.findLast()
		// predicate(p) -> idx.find(null, null, p)
	}
}

...