Page History

The key word `Merge` means the process in CompactionUtils here. That is the part of read all source files and write them to the target

...

file.

The log is logged as below(the log operation is marked as orange).

1. select the source file
2. log the source file list one by one
3. log `sequence` flag
4. create the target file
5. log the target file
6. start merge process
7. if finish merge a device, log (`device` `writer.offset`)
8. if merge finished, generate *.resource file
9. writer.endFile
10. log `merge finished`
11. delete source files
12. delete the ${storagegroup}.compaction.log file

We write 3 sequence files(0-0-0.tsfile,1-1-0.tsfile,2-2-0.tsfile) with 10 device(root.compactionTest.device0,root.compactionTest.device1,...,root.compactionTest.device9) as an example

1、Finish all merge with no compaction log

...

source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9 
17535
root.compactionTest.device0 
35058
root.compactionTest.device3 
52581
root.compactionTest.device4 
70104
root.compactionTest.device1 
87627
root.compactionTest.device2 
105150
root.compactionTest.device7 
122673
root.compactionTest.device8 
140196
root.compactionTest.device5 
157719
root.compactionTest.device6 
175242
merge finished

as the `merge finished` log is written before the source files is deleted, so the files may not be deletedeleted. The recover works as below.

...

source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9 
17535
root.compactionTest.device0 
35058
root.compactionTest.device3 
52581
root.compactionTest.device4 
70104
root.compactionTest.device1 
87627
root.compactionTest.device2 
105150
root.compactionTest.device7 
122673
root.compactionTest.device8 
140196
root.compactionTest.device5 
157719
root.compactionTest.device6 
175242

The recover works as below.

...

source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/1-1-0.tsfile
source
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/2-2-0.tsfile
sequence
target
/Users/surevil/Desktop/workspace/incubator-iotdb/server/target/0-0-1.tsfile
root.compactionTest.device9 
17535
root.compactionTest.device0 
35058
root.compactionTest.device3 
52581
root.compactionTest.device4 
70104
root.compactionTest.device1 
87627
root.compactionTest.device2 
105150
root.compactionTest.device7 
122673
root.compactionTest.device8 
140196

The recover works as below.

...