Motivation

One of the most important capabilities that Apache Ignite provides is the ability to colocate. There are two aspects of colocation:

1. Data colocation, i.e. grouping entries together based on a certain criteria.
2. Compute colocation, i.e. sending a computation to a node where a particular set of data resides.

Also Apache Ignite 1.x/2.x uses the term affinity key and a user must use affinity key in order to colocate the data. Because the main goal is data colocation, the user should not operate by affinity key term. Instead, the colocation term should be used directly.

Colocation Syntax

Colocation by non-composite primary key

As a typical example, let’s assume we have the following schema:

CREATE TABLE Accounts (
    account_id INT,
    name       VARCHAR,
    balance    DOUBLE,
    PRIMARY KEY (account_id)
);

CREATE TABLE Transactions (
    tx_id      INT,
    account_id INT,
    amount     DOUBLE,
    PRIMARY KEY (tx_id)
);

Most of the operations on such a schema will be isolated within a single account. Examples:

1. Add a new transaction, and update the balance of the corresponding account.
2. Run through all the transactions for an account, and calculate the average amount.

To achieve better performance, we want each of these operations to execute on a single node. Therefore, we need to group all transactions belonging to the same account within the same Raft group (partition, range, etc). Corresponding records that represent the account itself should also reside in the same group.

To achieve this, we want to perform affinity mapping for the Transactions table using the account_id field (as opposed to the tx_id, as it would happen currently). We need to allow users to specify the field used for colocation. For example, like this:

CREATE TABLE Transactions (
    tx_id      INT,
    account_id INT,
    amount     DOUBLE,
    PRIMARY KEY (tx_id, account_id)
)
COLOCATE BY (account_id);

Note: The affinity key must always be a part of the primary key. Otherwise, an exception is thrown. 

Note: Primary key columns, and therefore affinity key columns, MUST NOT permit NULL values according to the SQL standard.

Colocation by composite primary key

In case of using the composite primary key the tables also could be colocated. For example:

CREATE TABLE Orders (
    order_id      INT,
    department_id INT,
    create_date   TIMESTAMP,
    PRIMARY KEY (order_id, department_id)
);

CREATE TABLE OrderItems (
  order_item_id      INT,
  order_id           INT,
  department_id      INT,
  amount             INT,
  PRIMARY KEY (order_item_id)
);

For such data model the OrderItems table could be colocated with the Orders table in the following way:

CREATE TABLE OrderItems (
    order_item_id      INT,
    order_id           INT,
    department_id      INT,
    amount             INT,
    PRIMARY KEY (order_item_id, order_id, department_id)
)
COLOCATE BY (order_id, department_id);

Note: COLOCATE BY clause of colocated table (OrderItems table in the example above) must contain the same set of columns and in the same order as PRIMARY KEY clause of the main table (Orders table in the example above) in order to properly colocate the data.

Additional Requirements

Apache Ignite uses partitioning by a hash function. The hash function should have relatively even distribution. Because of it we can’t use primitive and very fast commutative hash functions (e.g. based on XOR). So the hash function should respect the order of fields that are included into the affinity key.

Affected APIs

Note: All APIs mentioned below are out of scope of data colocation feature design and will be designed later.

Table Groups

Note: this API will be defined later. Naming and ideas are not final.

Apache Ignite 3 will also provide the possibility to choose an affinity function for a group of some tables. At this point the term affinity [function] still should be used.

Compute API

The only API that will be affected is the Compute API where affinityRun() and affinityCall() methods will be renamed to executeColocated() (at the moment only one method is enough). 

Affinity Key Mapper

The affinity key mapper in Apache Ignite 1.x/2.x allows to implement some user defined logic for data colocation. Any user’s code is a potential problem because it leads to code deployment/redeployment issues, to error prone approach, etc. Instead of the affinity key mapper interface Apache Ignite 3 should provide the possibility to use a predefined set of SQL functions in the COLOCATE BY clause (see for example PARTITION BY clause in different databases).

Open Tickets

Closed Tickets