LevelCompaction recover scene analysis

The key word `Merge` means the process in CompactionUtils here. That is the part of read all source files and write them to the target file.

The log is logged as below.

1. select the source file
2. log the source file list one by one
3. log `sequence` flag
4. create the target file
5. log the target file
6. start merge process
7. if finish merge a device, log (`device` `writer.offset`)
8. log `merge finished`
9. delete source files
10. delete the ${storagegroup}.compaction.log file

We write 3 sequence files(0-0-0.tsfile,1-1-0.tsfile,2-2-0.tsfile) with 10 device(root.compactionTest.device0,root.compactionTest.device1,...,root.compactionTest.device9) as an example

1、Finish all merge with no compaction log

Just skip the recover of LevelCompaction.

2、Finish all merge but has compaction log remain

The log is below in the ${storagegroup}.compaction.log file.

source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9
17535
root.compactionTest.device0
35058
root.compactionTest.device3
52581
root.compactionTest.device4
70104
root.compactionTest.device1
87627
root.compactionTest.device2
105150
root.compactionTest.device7
122673
root.compactionTest.device8
140196
root.compactionTest.device5
157719
root.compactionTest.device6
175242
merge finished

as the `merge finished` log is written before the source files is deleted, so the files may not be delete. The recover works as below.

1. try to delete the source files if exists
2. delete the ${storagegroup}.compaction.log file

3、Finish all merge but no `merge finish` log

The log is below in the ${storagegroup}.compaction.log file.

source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9
17535
root.compactionTest.device0
35058
root.compactionTest.device3
52581
root.compactionTest.device4
70104
root.compactionTest.device1
87627
root.compactionTest.device2
105150
root.compactionTest.device7
122673
root.compactionTest.device8
140196
root.compactionTest.device5
157719
root.compactionTest.device6
175242

The recover works as below.

1. new RestorableTsFIleIOWriter writer
2. use the writer.hasCrashed() to detected whether the file has been sealed
3. (it is sealed in this condition) close the writer
4. delete the source files
5. delete the ${storagegroup}.compaction.log file

4、Merge not finished

The log is below in the ${storagegroup}.compaction.log file.

source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9
17535
root.compactionTest.device0
35058
root.compactionTest.device3
52581
root.compactionTest.device4
70104
root.compactionTest.device1
87627
root.compactionTest.device2
105150
root.compactionTest.device7
122673
root.compactionTest.device8
140196

The recover works as below.

1. new RestorableTsFIleIOWriter writer
2. use the writer.hasCrashed() to detected whether the file has been sealed
3. (it is un sealed in this condition) truncate the writer by the last offset logged
4. resume the merge process(skip the device set logged)
5. delete the source files
6. delete the ${storagegroup}.compaction.log file

5、Merge just started with no (device-offset) log

The log is below in the ${storagegroup}.compaction.log file.

source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile

as the device set is empty. The recover works as below.

1. delete the target file
2. delete the ${storagegroup}.compaction.log file

6、Just log all source files and sequence tag

The log is below in the ${storagegroup}.compaction.log file.

source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence

The recover works as below.

1. delete the ${storagegroup}.compaction.log file

7、Just log all source files

The log is below in the ${storagegroup}.compaction.log file.

source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile

The recover works as below.

1. delete the ${storagegroup}.compaction.log file

8、Just log part of source files

The log is below in the ${storagegroup}.compaction.log file.

source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile\

The recover works as below.

1. delete the ${storagegroup}.compaction.log file

9、Just create an empty log file

The log is below in the ${storagegroup}.compaction.log file.

The recover works as below.

1. delete the ${storagegroup}.compaction.log file