The key word `Merge` means the process in CompactionUtils here. That is the part of read all source files and write them to the target file.
The log is logged as below.
- select the source file
- log the source file list one by one
- log `sequence` flag
- create the target file
- log the target file
- start merge process
- if finish merge a device, log (`device` `writer.offset`)
- generate *.resource file
- writer.endFile
- log `merge finished`
- delete source files
- delete the ${storagegroup}.compaction.log file
We write 3 sequence files(0-0-0.tsfile,1-1-0.tsfile,2-2-0.tsfile) with 10 device(root.compactionTest.device0,root.compactionTest.device1,...,root.compactionTest.device9) as an example
1、Finish all merge with no compaction log
Just skip the recover of LevelCompaction.
2、Finish all merge but has compaction log remain
The log is below in the ${storagegroup}.compaction.log file.
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9
17535
root.compactionTest.device0
35058
root.compactionTest.device3
52581
root.compactionTest.device4
70104
root.compactionTest.device1
87627
root.compactionTest.device2
105150
root.compactionTest.device7
122673
root.compactionTest.device8
140196
root.compactionTest.device5
157719
root.compactionTest.device6
175242
merge finished
as the `merge finished` log is written before the source files is deleted, so the files may not be deleted. The recover works as below.
- try to delete the source files if exists
- delete the ${storagegroup}.compaction.log file
3、Finish all merge but no `merge finish` log
The log is below in the ${storagegroup}.compaction.log file.
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9
17535
root.compactionTest.device0
35058
root.compactionTest.device3
52581
root.compactionTest.device4
70104
root.compactionTest.device1
87627
root.compactionTest.device2
105150
root.compactionTest.device7
122673
root.compactionTest.device8
140196
root.compactionTest.device5
157719
root.compactionTest.device6
175242
The recover works as below.
- new RestorableTsFIleIOWriter writer
- use the writer.hasCrashed() to detected whether the file has been sealed
- (it is sealed in this condition) close the writer
- delete the source files
- delete the ${storagegroup}.compaction.log file
4、Merge not finished
The log is below in the ${storagegroup}.compaction.log file.
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9
17535
root.compactionTest.device0
35058
root.compactionTest.device3
52581
root.compactionTest.device4
70104
root.compactionTest.device1
87627
root.compactionTest.device2
105150
root.compactionTest.device7
122673
root.compactionTest.device8
140196
The recover works as below.
- new RestorableTsFIleIOWriter writer
- use the writer.hasCrashed() to detected whether the file has been sealed
- (it is un sealed in this condition) truncate the writer by the last offset logged
- resume the merge process(skip the device set logged)
- delete the source files
- delete the ${storagegroup}.compaction.log file
5、Merge just started with no (device-offset) log
The log is below in the ${storagegroup}.compaction.log file.
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
as the device set is empty. The recover works as below.
- delete the target file
- delete the ${storagegroup}.compaction.log file
6、Just log all source files and sequence tag
The log is below in the ${storagegroup}.compaction.log file.
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
The recover works as below.
- delete the ${storagegroup}.compaction.log file
7、Just log all source files
The log is below in the ${storagegroup}.compaction.log file.
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
The recover works as below.
- delete the ${storagegroup}.compaction.log file
8、Just log part of source files
The log is below in the ${storagegroup}.compaction.log file.
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile\
The recover works as below.
- delete the ${storagegroup}.compaction.log file
9、Just create an empty log file
The log is below in the ${storagegroup}.compaction.log file.
The recover works as below.
- delete the ${storagegroup}.compaction.log file