...

[1] https://mariadb.com/kb/en/library/table-pullout-optimization/
[2] https://dev.mysql.com/doc/refman/8.0/en/subquery-optimization-with-exists.html

Partition Pruning on Thin Clients

...

Thin Clients

Partitions could be calculated not only on the server side, but on clients as well. This way thin clients could be able to send requests directly to affected nodes, reducing latency. To achieve this we first define serializable partition tree model:

interface PartitionNode {
    Collection<Integer> apply(Object[] args);
}


class PartitionGroup implements PartitionNode {
    Collection<Object> parts; // Concrete partitions, arguments or both.
}


class PartitionExpression implements PartitionNode {
    PartitionNode left;
    PartitioNode right;
}

Partition tree is enriched with {{AffinityTopologyVersion}} it was built on, and affinity function descriptor. Descriptor can only be defined for well-known affinity functions, such as {{RendezvousAffinityFunction}}, and defines the logic on how to convert an object to partition. 

class PartitionInfo {
    PartitionNode tree;
    AffintiyTopologyVersion affTopVer;
    AffinityFunctionDecriptor affFunc;
}

When client executes request for the first time, the following sequence of actions happen:

1. Query request is received by the server
2. Partition tree is built using specific affinity topology version.
3. Query is executed as usual
4. If partition tree exists and is built with well-known affinity function, then it is attached to query response.

Client saves partition tree and re-use it for further requests as follows:

1. Query arguments are applied on the client. 
2. Target node is determined from the list of partitions. We assume that partition distribution for the given affinity topology versions has been requested in advance similarly how we do that for C++ thin client. 
3. If only one node is resolved, send request to it. If several nodes are resolved - send request to random node from the list. 
4. Request is executed on the server and current affinity topology version is attached to the response. If it differs from the one received from the client, new partition tree is built and attached.
5. Client checks if current affinity topology version differs. If yes - old partition tree is invalidated.

Tickets