You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

IDIEP-112
Author
Sponsor
Created

  

Status
DRAFT


Motivation

A criteria API represent an abstract paginated query to be executed on a cache. Supporting criteria API queries may result in several benefits:

  • It may significantly improve UX, by reducing amount of code required to iterate over cache entries
  • It allow to move filtration logic to the data nodes
  • Same API can be used in other parts of code

Scan and Index Queries are types of queries supported in Apache Ignite 2.x

Description

Criteria Query is logically just a facade for SQL – same way as Criteria API is a facade for SQL in JPA. The easiest way to implement this feature then is to just generate SQL queries.

There are different gateways to a table called Table Views. Currently we have RecordView and KeyValueView. Each view should have the Criteria Query API. The only difference is the type of objects the query result maps to.

The flow:

  1. The user supplies a Criteria and a QueryOptions.
  2. The client (or the API facade in embedded mode) generates the SELECT query based on the Criteria and sets the options (for now – just pageSize) in the query properties.
  3. The client runs the SQL as usual, potentially via the public API.
  4. The client obtains the ResultSet and turns it into an Iterator; it uses the view-defined Mappers to turn the Tuple into the target types.

Suggested Predicates List for Criteria API

  • Comparisons: =, >, <, >=, <=, IN
  • Combination: AND, OR, NOT
  • Nulls: IS NULL, IS NOT NULL

Criteria query works with transactions similarly to SQL

  • Query methods accept a transaction instance
  • If transaction is null, an implicit read-only transaction is opened

Configuration options

  • Ability to enforce an index(indexHint) to be used

  • Ability to configure pageSize for query. This is useful for tuning

API


Proposed API
/**
 * Represents a predicate. Implementations of this interface are basic building blocks for performing criteria queries.
 */
public interface Criteria {
    /**
     * Create a predicate for testing the column is equal to a given value.
     *
     * @param columnName Column name.
     * @param value Column value.
     * @return the created <b>equal</b> predicate instance.
     */
    static <R> Criteria equal(String columnName, Comparable<R> value) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }

    /**
     * Create a predicate for testing whether the column is greater than value.
     *
     * @param columnName Column name.
     * @param value Column value.
     * @return the created <b>greaterThan</b> predicate instance.
     */
    static <R> Criteria greaterThan(String columnName, Comparable<R> value) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }

    /**
     * Create a predicate for testing whether the column is greater than or equal to value.
     *
     * @param columnName Column name.
     * @param value Column value.
     * @return the created <b>greaterThanOrEqualTo</b> predicate instance.
     */
    static <R> Criteria greaterThanOrEqualTo(String columnName, Comparable<R> value) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }

    /**
     * Create a predicate for testing whether the column is less than value.
     *
     * @param columnName Column name.
     * @param value Column value.
     * @return the created <b>lessThan</b> predicate instance.
     */
    static <R> Criteria lessThan(String columnName, Comparable<R> value) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }

    /**
     * Create a predicate for testing whether the column is less than or equal to value.
     *
     * @param columnName Column name.
     * @param value Column value.
     * @return the created <b>lessThanOrEqualTo</b> predicate instance.
     */
    static <R> Criteria lessThanOrEqualTo(String columnName, Comparable<R> value) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }

    /**
     * Create a predicate for testing whether the first argument is an element of a certain collection.
     *
     * @param columnName Column name.
     * @param values Column values.
     * @return the created <b>in</b> predicate instance.
     */
    static <R> Criteria in(String columnName, Comparable<R>... values) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }

    /**
     * Creates a predicate that will perform the logical <b>and</b> operation on the given {@code predicates}.
     *
     * @param criterions the child predicates to form the resulting <b>and</b> predicate from.
     * @return the created <b>and</b> predicate instance.
     */
    static Criteria and(Criteria... criterions) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }

    /**
     * Creates a predicate that will perform the logical <b>or</b> operation on the given {@code predicates}.
     *
     * @param criterions the child predicates to form the resulting <b>or</b> predicate from.
     * @return the created <b>or</b> predicate instance.
     */
    static Criteria or(Criteria... criterions) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }

    /**
     * Creates a predicate that will negate the result of the given {@code predicate}.
     *
     * @param criteria the predicate to negate the value of.
     * @return the created <b>not</b> predicate instance.
     */
    static Criteria not(Criteria criteria) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }

    /**
     * Creates a predicate that will add SQL to where clause.
     *
     * @param sql Regular SQL where clause.
     * @param arguments Arguments for the statement.
     * @return the created <b>sql</b> predicate instance.
     */
    static Criteria sql(String sql, @Nullable Object... arguments) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }
}

/**
 * Criteria query result cursor. Implements {@link Iterable} only for convenience, e.g. {@link #iterator()}
 * can be obtained only once. Also if iteration is started then {@link #getAll()} method calls are prohibited.
 */
public interface CriteriaQueryCursor<T> extends Iterator<T>, Iterable<T>, AutoCloseable {
    /**
     * Gets all query results and stores them in the collection.
     * Use this method when you know in advance that query result is
     * relatively small and will not cause memory utilization issues.
     * <p>Since all the results will be fetched, all the resources will be closed
     * automatically after this call, e.g. there is no need to call {@link #close()} method in this case.
     *
     * @return List containing all query results.
     */
    default List<T> getAll() {
        return stream().collect(toList());
    }

    /** {@inheritDoc} */
    @Override
    default Iterator<T> iterator() {
        return this;
    }

    /**
     * Returns a sequential Stream over the elements covered by this cursor.
     *
     * @return Sequential Stream over the elements covered by this cursor.
     */
    default Stream<T> stream() {
        return StreamSupport.stream(spliterator(), false);
    }

    /**
     * Closes all resources related to this cursor. If the query execution is in progress
     * (which is possible in case of invoking from another thread), a cancel will be attempted.
     * Sequential calls to this method have no effect.
     * <p>Note: don't forget to close query cursors. Not doing so may lead to various resource leaks.
     */
    @Override void close();
}

/**
 * Represents a criteria builder. Implementations of this interface are basic building blocks for performing criteria queries.
 */
public interface CriteriaBuilder extends Criteria {
    static ColumnObject columnName(String columnName) {
        throw new UnsupportedOperationException("Not implemented yet.");
    }

    CriteriaBuilder and(Criteria criteria);

    CriteriaBuilder or(Criteria criteria);

    /**
     * Represents a column . Implementations of this interface are basic building blocks for performing criteria queries.
     */
    interface ColumnObject {
        /**
         * Create a predicate for testing the column is equal to a given value.
         *
         * @param value Column value.
         * @return the created <b>equal</b> predicate instance.
         */
        <R> CriteriaBuilder equal(Comparable<R> value);

        /**
         * Create a predicate for testing whether the column is greater than value.
         *
         * @param value Column value.
         * @return the created <b>greaterThan</b> predicate instance.
         */
        <R> CriteriaBuilder greaterThan(Comparable<R> value);

        /**
         * Create a predicate for testing whether the column is greater than or equal to value.
         *
         * @param value Column value.
         * @return the created <b>greaterThanOrEqualTo</b> predicate instance.
         */
        <R> Criteria greaterThanOrEqualTo(Comparable<R> value);

        /**
         * Create a predicate for testing whether the column is less than value.
         *
         * @param value Column value.
         * @return the created <b>lessThan</b> predicate instance.
         */
        <R> Criteria lessThan(Comparable<R> value);

        /**
         * Create a predicate for testing whether the column is less than or equal to value.
         *
         * @param value Column value.
         * @return the created <b>lessThanOrEqualTo</b> predicate instance.
         */
        <R> Criteria lessThanOrEqualTo(Comparable<R> value);

        /**
         * Create a predicate for testing whether the first argument is an element of a certain collection.
         *
         * @param values Column values.
         * @return the created <b>in</b> predicate instance.
         */
        <R> Criteria in(String columnName, Comparable<R>... values);
    }
}

/**
 * Criteria query options.
 */
public class CriteriaQueryOptions {
    /** Default options. */
    public static final CriteriaQueryOptions DEFAULT = builder().build();

    private final int pageSize;

    /**
     * Constructor.
     *
     * @param pageSize Page size.
     */
    private CriteriaQueryOptions(int pageSize) {
        this.pageSize = pageSize;
    }

    /**
     * Creates a new builder.
     *
     * @return Builder.
     */
    public static Builder builder() {
        return new Builder();
    }

    /**
     * Gets a page size - the maximum number of result rows that can be fetched at a time.
     *
     * @return Batch size.
     */
    public int pageSize() {
        return pageSize;
    }

    /**
     * Builder.
     */
    public static class Builder {
        private int pageSize = 1000;

        /**
         * Sets the page size (the number of entries that will be sent to the cluster in one network call).
         *
         * @param pageSize Page size.
         * @return This builder instance.
         */
        public Builder pageSize(int pageSize) {
            if (pageSize <= 0) {
                throw new IllegalArgumentException("Page size must be positive: " + pageSize);
            }

            this.pageSize = pageSize;

            return this;
        }

        /**
         * Builds the options.
         *
         * @return Criteria query options.
         */
        public CriteriaQueryOptions build() {
            return new CriteriaQueryOptions(pageSize);
        }
    }
}

/**
 * Represents an object which can be queried with criteria.
 *
 * @param <T> Entry type.
 */
public interface CriteriaQuerySource<T> {
    /**
     * Criteria query over cache entries.
     *
     * @param tx Transaction to execute the query within or {@code null}.
     * @param criteria If {@code null} then all entries will be returned.
     * @throws SqlException If failed.
     */
    default CriteriaQueryCursor<T> queryCriteria(@Nullable Transaction tx, @Nullable Criteria criteria) {
        return queryCriteria(tx, criteria, CriteriaQueryOptions.DEFAULT);
    }

    /**
     * Criteria query over cache entries.
     *
     * @param tx Transaction to execute the query within or {@code null}.
     * @param criteria If {@code null} then all entries will be returned.
     * @param opts Criteria query options.
     * @throws SqlException If failed.
     */
    CriteriaQueryCursor<T> queryCriteria(@Nullable Transaction tx, @Nullable Criteria criteria, CriteriaQueryOptions opts);

    /**
     * Execute criteria query over cache entries in an asynchronous way.
     *
     * @param tx Transaction to execute the query within or {@code null}.
     * @param criteria If {@code null} then all entries will be returned.
     * @return Operation future.
     * @throws SqlException If failed.
     */
    default CompletableFuture<CriteriaQueryCursor<T>> queryCriteriaAsync(@Nullable Transaction tx, @Nullable Criteria criteria) {
        return queryCriteriaAsync(tx, criteria, CriteriaQueryOptions.DEFAULT);
    }

    /**
     * Execute criteria query over cache entries in an asynchronous way.
     *
     * @param tx Transaction to execute the query within or {@code null}.
     * @param criteria If {@code null} then all entries will be returned.
     * @param opts Criteria query options.
     * @return Operation future.
     * @throws SqlException If failed.
     */
    CompletableFuture<CriteriaQueryCursor<T>> queryCriteriaAsync(
            @Nullable Transaction tx,
            @Nullable Criteria criteria,
            CriteriaQueryOptions opts
    );
}

public interface KeyValueView<K, V> extends DataStreamerTarget<Entry<K, V>>, CriteriaQuerySource<Entry<K, V>> {
    // ...
}

public interface RecordView<R> extends DataStreamerTarget<R>, CriteriaQuerySource<R> {
    // ...
}


Usage examples
// Criteria
try (var cursor = kvView().queryCriteria(
        null,
        and(equal("intCol", 42), greaterThan("primitiveIntCol", 9000), equal("booleanCol", true)),
        CriteriaQueryOptions.builder().pageSize(10).build()
)) {
    // ...
}

// CriteriaBuilder
try (var cursor = kvView().queryCriteria(
        null,
        columnName("intCol").equal(42)
                .and(columnName("primitiveIntCol").greaterThan(9000))
                .and(columnName("booleanCol").equal(true))
)) {
     // ... 
}

//  SQL where clause as criteria.
try (var cursor = kvView().queryCriteria(null, sql("intCol = ? AND primitiveIntCol > ? AND booleanCol = true", 42, 9000))) {
     // ... 
}

Risks and Assumptions

Expected to have similar performance for single-key criteria query, single-key SQL, table.get(PK). At the same time, SQL parsing may take much longer than expected.

Reactive API should be added for both SQL and Criteria, so it's not a point of this IEP.

USE INDEX-style hint looks like is not supported at this moment, but we still can add API and SQL generation for it.

NOTE: Comparing with table.get(PK) should be done with disabled partition awareness

Future improvement for criteria query

  • Lookup affinity node if PK or colocation key is in the criteria.
  • Create cache of prepared statement used in criteria query

Reference Links

Scan Query in Apache Ignite 2.x: https://ignite.apache.org/docs/latest/key-value-api/using-cache-queries#executing-scan-queries.

Index Query in Apache Ignite 2.x: https://ignite.apache.org/docs/latest/key-value-api/using-cache-queries#executing-index-queries.

Tickets

Unable to render Jira issues macro, execution error.

  • No labels