The key word `Merge` means the process in CompactionUtils here. That is the part of read all source files and write them to the target file.
The log is logged as below(the log operation is marked as orange).
- select the source file
- log the source file list one by one
- log `sequence` flag
- create the target file
- log the target file
- start merge process
- if finish merge a device, log (`device` `writer.offset`)
- if merge finished, generate *.resource file
- writer.endFile
- log `merge finished`
- delete source files
- delete the ${storagegroup}.compaction.log file
We write 3 sequence files(0-0-0.tsfile,1-1-0.tsfile,2-2-0.tsfile) with 10 device(root.compactionTest.device0,root.compactionTest.device1,...,root.compactionTest.device9) as an example
1、Finish all merge with no compaction log
...
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9
17535
root.compactionTest.device0
35058
root.compactionTest.device3
52581
root.compactionTest.device4
70104
root.compactionTest.device1
87627
root.compactionTest.device2
105150
root.compactionTest.device7
122673
root.compactionTest.device8
140196
root.compactionTest.device5
157719
root.compactionTest.device6
175242
merge finished
as the `merge finished` log is written before the source files is deleted, so the files may not be deletedeleted. The recover works as below.
...
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9
17535
root.compactionTest.device0
35058
root.compactionTest.device3
52581
root.compactionTest.device4
70104
root.compactionTest.device1
87627
root.compactionTest.device2
105150
root.compactionTest.device7
122673
root.compactionTest.device8
140196
root.compactionTest.device5
157719
root.compactionTest.device6
175242
The recover works as below.
...
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9
17535
root.compactionTest.device0
35058
root.compactionTest.device3
52581
root.compactionTest.device4
70104
root.compactionTest.device1
87627
root.compactionTest.device2
105150
root.compactionTest.device7
122673
root.compactionTest.device8
140196
The recover works as below.
...