...
First get all the data files ending with .tsfile in the virtual storage group and return corresponding TsFileResource object. File types are as follows:
- Sequence Files
- new version: 0.10 version tsfiles(sealed/unsealed)
- old version: 0.9 version tsfiles(sealed)
- Unsequence Files
- new version: 0.10 version tsfiles(sealed/unsealed)
- old version: 0.9 version tsfiles(sealed)
- Sequence Files
If 0.9 version TsFile exists in the storage group, add the old version's sequence and unsequence files to
upgradeSeqFileList
andupgradeSeqFileList
respectively for upgrade and query.Group sequence and unsequence files according to time partition id and store as
Map<Long, List<TsFileResource>>.
- Call the recoverTsFiles method to recover all sequence/unsequence files of each time partition
- Check whether there is a Modification file during the merge, and call the
RecoverMergeTask.recoverMerge
method to recover the merge. - Call the recoverCompaction method to recover the compaction.
Traverse all sequence/unsequence tsfiles(including old version files) and call the updatePartitionFileVersion method on them to update the version number of each time partition.
Call the
updateLastestFlushedTime()
method to update thelatestTimeForEachDevice
,partitionLatestFlushedTimeForEachDevice
andglobalLatestFlushedTimeForEachDevice
with old version sequential tsfile.latestTimeForEachDevice
records the latest timestamp under each partition that all devices have been inserted into (including unflushed and flushed)partitionLatestFlushedTimeForEachDevice
records the latest timestamp of all devices under each partition that has been flushed. It is used to determine whether a newly inserted point is out of order.globalLatestFlushedTimeForEachDevice
records the latest timestamp of all devices that have been flushed (a summary of the latest timestamps of each partition)
Finally traverse all restored sequence files to update
latestTimeForEachDevice
,partitionLatestFlushedTimeForEachDevice
andglobalLatestFlushedTimeForEachDevice
again
Recover
...
all TsFiles(Seq/Unseq)
...
in one partiton
org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor.recoverTsFiles(List<TsFileResource> tsFiles, boolean isSeq)
...
Construct a
TsFileRecoverPerformer
object to recover the TsFile- Call the recover() method of TsFileRecoverPerformer, whether to redo wal is judged by which level the tsfile is at
- If the tsfile isn‘t at the level 0, the recover() method doesn't need redoing wal, just adds the restored TsFileResource to TsFileManagement and skip subsequent procedures.
- If the tsfile is at the level 0, the recover() method needs redoing wal and execute subsequent procedures.
- Whether to construct TsFileProcessor is judged by if the tsfile is the last file.
If the tsfile is not the last file or cannot be written into, just set the
closed
attribute of theTsFileResource
totrue
.If the tsfile is the last file and can be written into, it means that this is the last TsFile of this partition, and it is unsealed, so keep it unsealed and construct a
TsFileProcessor
object for it, then place the TsFileProcessor object in workSequenceTsFileProcessors or workUnsequenceTsFileProcessors.
Finally, add the
TsFileResource
object into TsFileManagement.
...
Recover one TsFile
org.apache.iotdb.db.writelog.recover.TsFileRecoverPerformer.recover(boolean needRedoWal, Supplier<ByteBuffer[]> supplier, Consumer<ByteBuffer[]> consumer)
...