Motivation

Replicated zones are zones with partitions that have replicas on every node of the cluster that matches the zone filter and storage profile filter. These zones can provide better performance on SQL joins. 

Currently in Apache Ignite 3, we have an ability to set any number of replicas for any zone regardless of its initial replica number. The problem is that in large clusters, if we create a zone with a number of replicas that would be equal to the cluster size, there will be too many members in each replication group to maintain consensus. This will have a serious impact on performance. Moreover, we don’t have automatic scale up for this case: if the user decides to scale up their cluster, they would need to change the replica number manually (unless it is big enough, exceeding the nodes number).

Therefore, we need special options for zones to create members of replication groups that would not be a part of consensus, and to be able to scale the zones along with the cluster.

Description

Requirements

• user should be able to create a zone with partitions that have replicas on every node of the cluster that matches the zone filter and storage profiles;
• replicas should be operational in a case when they are not included into the consensus group, let’s call them learners. Learners are created on every node that matches the zone filters where regular replica is not created;
• read-write transactions work as on regular zones (i.e. through primary replicas);
• read-only transactions and SQL within read-only transactions use learners as regular non-primary replicas;
• filters and storage profiles for the zones having learners should work in the same way as for regular zones;
• auto scaling: if new nodes are included into the cluster, learners are created there automatically (if the nodes match the zone filters).

Proposal

Reference links

Tickets

IGNITE-23790 - Getting issue details... STATUS