IDIEP-115
Author
Sponsor
Created

  

Status
DRAFT


Motivation

A criteria API represent an abstract paginated query to be executed on a cache. Supporting criteria API queries may result in several benefits:

  • It may significantly improve UX, by reducing amount of code required to iterate over cache entries
  • It allow to move filtration logic to the data nodes
  • Same API can be used in other parts of code

Scan and Index Queries are types of queries supported in Apache Ignite 2.x

Description

Criteria Query is logically just a facade for SQL – same way as Criteria API is a facade for SQL in JPA. The easiest way to implement this feature then is to just generate SQL queries.

There are different gateways to a table called Table Views. Currently we have RecordView and KeyValueView. Each view should have the Criteria Query API. The only difference is the type of objects the query result maps to.

The flow:

  1. The user supplies a Criteria and a QueryOptions.
  2. The client (or the API facade in embedded mode) generates the SELECT query based on the Criteria and sets the options (for now – just pageSize) in the query properties.
  3. The client runs the SQL as usual, potentially via the public API.
  4. The client obtains the ResultSet and turns it into an Iterator; it uses the view-defined Mappers to turn the Tuple into the target types.

Suggested Predicates List for Criteria API

  • Comparisons: =, >, <, >=, <=, IN
  • Combination: AND, OR, NOT
  • Nulls: IS NULL, IS NOT NULL

Criteria query works with transactions similarly to SQL

  • Query methods accept a transaction instance
  • If transaction is null, an implicit read-only transaction is opened

Configuration options

  • Ability to enforce an index(indexHint) to be used

  • Ability to configure pageSize for query. This is useful for tuning

API


Proposed API
/**
 * Represents a predicate. Implementations of this interface are basic building blocks for performing criteria queries.
 */
public interface Criteria {
    /**
     * Creates a predicate that tests whether the column value is equal to the given condition.
     *
     * @param columnName Column name.
     * @param condition Target condition. 
     */
    static Expression columnValue(String columnName, Condition condition);
 
    /**
     * Creates a condition that test the examined object is equal to the specified {@code value}.
     *
     * @param value Target value.
     */
    static <R> Condition equalTo(Comparable<R> value);

    /**
     * Creates a condition that test the examined object is equal to the specified {@code value}.
     *
     * @param value Target value.
     */
    static Condition equalTo(byte[] value);

    /**
     * Creates a condition that test the examined object is not equal to the specified {@code value}.
     *
     * @param value Target value.
     */
    static <R> Condition greaterThan(Comparable<R> value);

    /**
     * Creates a condition that test the examined object is greater than or equal than the specified {@code value}.
     *
     * @param value Target value.
     */
    static <R> Condition greaterThanOrEqualTo(Comparable<R> value);

    /**
     * Creates a condition that test the examined object is less than the specified {@code value}.
     *
     * @param value Column value.
     */
    static <R> Condition lessThan(Comparable<R> value);

    /**
     * Creates a condition that test the examined object is less than or equal than the specified {@code value}.
     *
     * @param value Column value.
     */
    static <R> Condition lessThanOrEqualTo(Comparable<R> value);

    /**
     * Creates a condition that test the examined object is is found within the specified {@code collection}.
     *
     * @param values Column values.
     * @return the created <b>in</b> predicate instance.
     */
    static <R> Condition in(Comparable<R>... values);

    /**
     * Creates a condition that test the examined object is is found within the specified {@code collection}.
     *
     * @param values Column values.
     * @return the created <b>in</b> predicate instance.
     */
    static <R> Condition in(byte[]... values);

    /**
     * Creates the {@code and} of the expressions.
     *
     * @param expressions Expressions.
     */
    static Expression and(Expression... expressions);

    /**
     * Creates the {@code or} of the expressions.
     *
     * @param expressions Expressions.
     */
    static Expression or(Expression... expressions);

    /**
     * Creates the negation of the predicate.
     *
     * @param expression Expression.
     */
    static Expression not(Expression expression);
}

/**
 * Provides methods to iterate over operation results and release underlying resources.
 */
public interface Cursor<T> extends Iterator<T>, AutoCloseable {
    /**
     * Closes this cursor and releases any underlying resources.
     */
    @Override void close();
}

/**
 * Provides methods to iterate over operation results and release underlying resources in an asynchronous way.
 */
public interface AsyncCursor<T> {
    /**
     * Returns the current page content if the operation returns results.
     */
    Iterable<T> currentPage();

    /**
     * Returns the current page size if the operation return results.
     */
    int currentPageSize();

    /**
     * Fetches the next page of results asynchronously.
     */
    CompletableFuture<? extends AsyncCursor<T>> fetchNextPage();

    /**
     * Indicates whether there are more pages of results.
     */
    boolean hasMorePages();

    /**
     * Closes this cursor and releases any underlying resources.
     */
    CompletableFuture<Void> closeAsync();
}

/**
 * Criteria query options.
 */
public class CriteriaQueryOptions {
    /** Default options. */
    public static final CriteriaQueryOptions DEFAULT = builder().build();

    private final int pageSize;

    /**
     * Constructor.
     *
     * @param pageSize Page size.
     */
    private CriteriaQueryOptions(int pageSize) {
        this.pageSize = pageSize;
    }

    /**
     * Creates a new builder.
     *
     * @return Builder.
     */
    public static Builder builder() {
        return new Builder();
    }

    /**
     * Gets a page size - the maximum number of result rows that can be fetched at a time.
     *
     * @return Batch size.
     */
    public int pageSize() {
        return pageSize;
    }

    /**
     * Builder.
     */
    public static class Builder {
        private int pageSize = 1000;

        /**
         * Sets the page size (the number of entries that will be sent to the cluster in one network call).
         *
         * @param pageSize Page size.
         * @return This builder instance.
         */
        public Builder pageSize(int pageSize) {
            if (pageSize <= 0) {
                throw new IllegalArgumentException("Page size must be positive: " + pageSize);
            }

            this.pageSize = pageSize;

            return this;
        }

        /**
         * Builds the options.
         *
         * @return Criteria query options.
         */
        public CriteriaQueryOptions build() {
            return new CriteriaQueryOptions(pageSize);
        }
    }
}

/**
 * Represents an object which can be queried with criteria.
 *
 * @param <T> Entry type.
 */
public interface CriteriaQuerySource<T> {
    /**
     * Criteria query over cache entries.
     *
     * @param tx Transaction to execute the query within or {@code null}.
     * @param criteria If {@code null} then all entries will be returned.
     * @throws SqlException If failed.
     */
    default Cursor<T> query(@Nullable Transaction tx, @Nullable Criteria criteria) {
        return query(tx, criteria, CriteriaQueryOptions.DEFAULT);
    }

    /**
     * Criteria query over cache entries.
     *
     * @param tx Transaction to execute the query within or {@code null}.
     * @param criteria If {@code null} then all entries will be returned.
     * @param opts Criteria query options.
     * @throws SqlException If failed.
     */
    Cursor<T> query(@Nullable Transaction tx, @Nullable Criteria criteria, @Nullable CriteriaQueryOptions opts);

    /**
     * Execute criteria query over cache entries in an asynchronous way.
     *
     * @param tx Transaction to execute the query within or {@code null}.
     * @param criteria If {@code null} then all entries will be returned.
     * @return Operation future.
     * @throws SqlException If failed.
     */
    default CompletableFuture<AsyncCursor<T>> queryAsync(@Nullable Transaction tx, @Nullable Criteria criteria) {
        return queryAsync(tx, criteria, CriteriaQueryOptions.DEFAULT);
    }

    /**
     * Execute criteria query over cache entries in an asynchronous way.
     *
     * @param tx Transaction to execute the query within or {@code null}.
     * @param criteria If {@code null} then all entries will be returned.
     * @param opts Criteria query options.
     * @return Operation future.
     * @throws SqlException If failed.
     */
    CompletableFuture<AsyncCursor<T>> queryAsync(
            @Nullable Transaction tx,
            @Nullable Criteria criteria,
            @Nullable CriteriaQueryOptions opts
    );
}

public interface KeyValueView<K, V> extends DataStreamerTarget<Entry<K, V>>, CriteriaQuerySource<Entry<K, V>> {
    // ...
}

public interface RecordView<R> extends DataStreamerTarget<R>, CriteriaQuerySource<R> {
    // ...
}


Usage examples
// Criteria
try (Cursor<Entry<Tuple, Tuple>> cursor = kvView().query(
        null,
        and(columnValue("intCol", equalTo(42)), columnValue("primitiveIntCol", greaterThan(9000)), columnValue("booleanCol", equalTo(true))),
        CriteriaQueryOptions.builder().pageSize(10).build()
)) {
    // ...
}

Risks and Assumptions

Expected to have similar performance for single-key criteria query, single-key SQL, table.get(PK). At the same time, SQL parsing may take much longer than expected.

Reactive API should be added for both SQL and Criteria, so it's not a point of this IEP.

USE INDEX-style hint looks like is not supported at this moment, but we still can add API and SQL generation for it.

NOTE: Comparing with table.get(PK) should be done with disabled partition awareness

Future improvement for criteria query

  • Lookup affinity node if PK or colocation key is in the criteria.
  • Create cache of prepared statement used in criteria query

Reference Links

Scan Query in Apache Ignite 2.x: https://ignite.apache.org/docs/latest/key-value-api/using-cache-queries#executing-scan-queries.

Index Query in Apache Ignite 2.x: https://ignite.apache.org/docs/latest/key-value-api/using-cache-queries#executing-index-queries.

Tickets

Unable to render Jira issues macro, execution error.

  • No labels