Page History

...

1. First get all the data files ending with .tsfile in the virtual storage group and return corresponding TsFileResource object. File types are as follows:

   • Sequence Files
     • new version: 0.10 version tsfiles(sealed/unsealed)
     • old version: 0.9 version tsfiles(sealed)
   • Unsequence Files
     • new version: 0.10 version tsfiles(sealed/unsealed)
     • old version: 0.9 version tsfiles(sealed)

2. If 0.9 version TsFile exists in the storage group, add the old version's sequence and unsequence files to upgradeSeqFileList and upgradeSeqFileList respectively for upgrade and query.

3. Group sequence and unsequence files according to time partition id and store as Map<Long, List<TsFileResource>>.

4. Call the recoverTsFiles method to recover all sequence/unsequence files of each time partition(details of this method will be explained in the next section) .
5. Check whether there is a Modification file during the merge, and call the RecoverMergeTask.recoverMerge method to recover the merge.
6. Call the recoverCompaction method to recover the compaction.

7. Traverse all sequence/unsequence tsfiles(including old version files) and call the updatePartitionFileVersion method on them to update the version number of each time partition.

8. Call the updateLastestFlushedTime() method to update the latestTimeForEachDevice, partitionLatestFlushedTimeForEachDevice and globalLatestFlushedTimeForEachDevice with old version sequential tsfile.

   • latestTimeForEachDevice records the latest timestamp under each partition that all devices have been inserted into (including unflushed and flushed)
   • partitionLatestFlushedTimeForEachDevice records the latest timestamp of all devices under each partition that has been flushed. It is used to determine whether a newly inserted point is out of order.
   • globalLatestFlushedTimeForEachDevice records the latest timestamp of all devices that have been flushed (a summary of the latest timestamps of each partition)

9. Finally traverse all restored sequence files to update latestTimeForEachDevice, partitionLatestFlushedTimeForEachDevice and globalLatestFlushedTimeForEachDevice again

Recover

...

all TsFiles(Seq/Unseq)

...

in one partiton

org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.recoverTsFiles(List<TsFileResource> tsFiles, boolean isSeq)

...

1. Construct a TsFileRecoverPerformer object to recover the TsFile

2. Call the recover() method of TsFileRecoverPerformer, whether to redo wal is judged by which level the tsfile is at(details of this method will be explained in the next section) .
   • If the tsfile isn‘t at the level 0, the recover() method doesn't need redoing wal, just adds the restored TsFileResource to TsFileManagement and skip subsequent procedures.
   • If the tsfile is at the level 0, the recover() method needs redoing wal and execute subsequent procedures.
3. Whether to construct TsFileProcessor is judged by if the tsfile is the last file.

   • If the tsfile is not the last file or cannot be written into, just set the closed attribute of the TsFileResource to true.

   • If the tsfile is the last file and can be written into, it means that this is the last TsFile of this partition, and it is unsealed, so keep it unsealed and construct a TsFileProcessor object for it, then place the TsFileProcessor object in workSequenceTsFileProcessors or workUnsequenceTsFileProcessors.

4. Finally, add the TsFileResource object into TsFileManagement.

...

Recover one TsFile

org.apache.iotdb.db.writelog.recover.TsFileRecoverPerformer.recover(boolean needRedoWal, Supplier<ByteBuffer[]> supplier, Consumer<ByteBuffer[]> consumer)

...