...

In the actual test, according to the above calculated device metadata memory occupation is only 20M. Considering that IoTDB metadata management uses related HashMap and other data structures, the memory occupied by IO reading and writing in Mlog, the upper limit of metadata memory is set to 1G in the actual scenario (the part that cannot be used up will also be used by other parts of the system). The metadata occupies 5% of the memory ratio in the heap, which is in line with the normal read and write memory load of IoTDB. There is no OOM phenomenon in the test and long test, which proves that the physical volume template does reduce the memory redundancy.

4. Set and Show Template

4.1 Function Definition

4.1.1 Set Template

A created template can be mounted to a node at any level of the metadata tree.
There is at most one mounted template on any path, i.e. if a node is mounted with a template, neither its ancestor node nor its descendant nodes can mount the template.
The subtree of this node can sense the mounted template and perform subsequent activation operations.
If the physical quantity names of existing time series on the subtree of the target path overlap with the physical quantity names in the template, mounting the template to that path is prohibited.

set schema template t1 to root.sg1.d1

4.1.2 Show Template

Queries the path where the specified template is mounted.

show paths set schema template t1

+-----------+
|      paths|
+-----------+
|root.sg1.d1|
+-----------+

4.2 Scenario

Template mounting operations should be small in practical scenarios, and it is generally recommended that users mount templates at the storage group level of the metadata tree or at nodes as high up as possible.
It is not recommended to abuse templates. Scenarios such as massive templates and massive mounts are not considered. It is enough to ensure that the system does not crash in this case, and low performance is normal.

4.3 Overview

4.3.1 Template mounting infomation

Since the logical metadata tree is sliced on the physical storage in distributed, the use of templates needs to be designed with specific slicing rules.
The rules for slicing the metadata tree in distributed are: first by storage group, then within the storage group then hash slots by device path, and each slot will have a schemaRegion to which it belongs. the rules have the following characteristics.

1. the nodes above the storage group in the metadata tree will be stored persistently in the configNode, and each dataNode will be cached in conjunction with the operation.
2. device nodes and physical nodes (leaf nodes) in the metadata tree have clear schemaRegion to which they belong, and there are no duplicate device nodes or physical nodes between different schemaRegions.
3. Intermediate nodes on the path between the metadata tree species storage group nodes and device nodes may have duplicate storage in different schemaRegion.
   
   

Image Added

As one of the metadata of the cluster as a whole, the template's mount is the information that multiple schemaRegions depend on together, and its consistency maintenance overhead is high because intermediate nodes will be stored repeatedly among different schemaRegions. Therefore, the template mount information is stored on the configNode as the unique standard of the cluster, and all the mount information is pushed to each dataNode for caching for use. Therefore, the management of template mount information follows the following architectural approach.

Image Added

4.3.2 Template mounting process

1. Pre-mount template information on ConfigNode, each DataNode caches and puts into use the formally mounted templates normally when it joins the cluster, and caches the pre-mounted templates in pre-release state.
2. ConfigNode pre-publishes the template mount information to all DataNodes, and DataNode caches it in pre-published state. If a DataNode executes abnormally during pre-publishing, it rolls back the pre-publishing and pre-mounting operations; if a DataNode is never connected, it keeps retrying until its status changes to Unknown; if all of them execute successfully, it executes subsequent operations.
3. ConfigNode checks whether there is any sequence with the same name as the physical quantity of the template in all related schemaRegion, if it exists, the template mounting operation fails and rolls back the pre-mounting and pre-publishing operation; if it does not exist, it continues to execute the subsequent process.
4. ConfigNode sets the pre-mounted template to the official mount state.
5. ConfigNode broadcasts to all DataNodes to commit the pre-release information, and DataNode puts the pre-release template mount information into normal use. If there is an abnormality in the execution of a DataNode during commit, all the template mounts are re-pushed to the DataNode according to the abnormality (i.e., the cache of the DataNode is reset); if a DataNode is never connected, it keeps retrying until its status changes to Unknown.

Once the commit phase is entered, the DataNode that has completed the commit operation may start to execute the template activation operation triggered by the data write. At this point, it is necessary to ensure that the template mount is officially put into use in the cluster, so the DataNode with the status of Running must ensure the final consistency of its template cache by retrying or resetting measures.

The entire process is shown in the following figure. To highlight the implementation, the following figure simplifies the communication process between each dataNode and configNode node, and only shows the execution process, in fact, all the judgment logic and branch selection in the following figure are executed in the configNode node.

Image Added

4.3.3 Concurrent handling of auto-registration and template mounts

When performing auto-registration, it is necessary to check whether there is a template mount situation, and then determine whether to perform a template activation operation or register a common sequence. There are several scenarios for this concurrency scenario as follows.

1. There is no formal or pre-release template mount information on the DataNode: The cluster has no template mount operation or is pre-releasing without notifying the current DataNode at this time, so the execution of auto-registration will perform the registration of ordinary sequences.
2. Pre-published template mount information on DataNode: The cluster may be in the commit execution stage without notifying the current DataNode, or the current DataNode does not receive the commit notification due to network problems, then the DataNode can initiate a request to the ConfigNode, and if the template mount information on the ConfigNode is in the pre-published state, the DataNode can initiate a request to the ConfigNode. If the information mounted on the ConfigNode is in the pre-release state, the pre-release information cannot be put into use; if it is in the formal mount state, the commit operation can be executed in advance (idempotent), and the information mounted on the template will be put into the template activation operation.
3. There is formally published template mount information on DataNode: put this information into use directly and execute template activation operation.
   

When DataNode performs the registration sequence, if the sequence to be registered is detected to overlap with the template mount information (prefix overlap, physical quantity renaming), then regardless of the pre-release or official release of the mount information detected at this time, the ordinary sequence registration is rejected to ensure that the subsequent template activation operation will not be impossible because of the overlapping sequence, and it is guaranteed that there will not be any sequence with the template under the template mount path The sequence with the same name as the physical quantity in the template mount path.

4.3.4 Concurrent handling of template mounts and new DataNodes

When performing template mounting operations, there may be cluster expansion and contraction, DataNode addition and deletion operations. Mainly consider the case of adding DataNode.

1. When a DataNode is added before the completion of pre-mounting, this DataNode only needs to synchronize the rest of the template information of formal mounting, and this pre-mounting information will be processed in the subsequent phase.
2. When a new DataNode is added in the pre-release phase, the DataNode will cache the pre-mounted information to the pre-release state when synchronizing the template mount information, as a remedy for the pre-release phase (it may get no new DataNode in the node information).
3. When a new DataNode is added in the commit phase, the template mount information is directly cached in the official state and put into use since it is already in the official state on the ConfigNode.
   
   

4.4 Detailed Design

Since templates are mounted on trees, the existing MTreeAboveSG can be extended to record the template mount information on it. MTreeAboveSG is aggregated from MNodes, so the template mount information will be stored in the specific MNode.

In order to decouple the information storage of the template itself from that of the metadata tree, the template is Id'd, so that only the mounted template Id is stored on the MNode, and if a specific template is used, the specific template is obtained from the TemplateManager based on the template Id.
The template Id is implemented as a self-incrementing Id. An int variable is maintained in memory, and each time a new template is created, the value of this variable will be used as the Id of the new template, and the variable will be added by one; when restarted and restored, this variable will be set to the maximum value of all current template Id + 1.

MTreeAboveSG will be renamed to ConfigMTree and provide the corresponding query interface for template mount information. configMTree will serialize the template Id of InternalMNode mount when it does snapshot.

Note: ConfigNode.TemplateManager only maintains the template information; the template mount information and part of the metadata tree hierarchy information are maintained by ClusterSchemaManager.
The related code can be found in the setSchemaTemplate method and getPathsSetTemplate method in SchemaRegionMemoryImpl.