Versions Compared

Old Version 30

changes.mady.by.user Lei Rui

Saved on

compared with

New Version 31

changes.mady.by.user Lei Rui

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

压缩方式GZIPLZ4SNAPPYUNCOMPRESSED
dataset/disk/rl/zc_data/ZT11529.csv/disk/rl/zc_data/ZT11529.csv/disk/rl/zc_data/ZT11529.csv/disk/rl/zc_data/ZT11529.csv
pagePointNum(ppn)10000100001000010000
numOfPagesInChunk(pic)100100100100
chunksWritten(cw)13131313
timeEncoding(te)TS_2DIFFTS_2DIFFTS_2DIFFTS_2DIFF
valueDataType(vt)DOUBLEDOUBLEDOUBLEDOUBLE
valueEncoding(ve)GORILLAGORILLAGORILLAGORILLA
compression(co)GZIPLZ4SNAPPYUNCOMPRESSED
totalPointNum12780287127802871278028712780287
tsfileSize(MB)19.3486261423.7774105123.1564121236.30773735
chunkDataSize_stats_mean(MB)1.5151396591.860070071.8131843412.837062438
compressedPageSize_stats_mean(B)15824.1666719440.27518948.6929684.7575
uncompressedPageSize_stats_mean(B)29684.757529684.757529684.757529684.7575
timeBufferSize_stats_mean(B)11461.462511461.462511461.462511461.4625
valueBufferSize_stats_mean(B)18221.2683318221.2683318221.2683318221.26833
[2] category: (A)get ChunkStatistic->(B)load on-disk Chunk->(C)get PageStatistics->(D)load in-memory PageData
[Avg&Per] (A)get_chunkMetadatas87316.85010000001 us - 8.19883087518919%100289.6416 us - 12.59103802007131%89466.6045 us - 11.199530364576052%88760.777 us - 10.335699109318087%
[Avg&Per] (B)load_on_disk_chunk160699.6025 us - 15.089285299443361%176784.88239999997 us - 22.19476647997349%105802.9384 us - 13.244531050378352%191518.85280000002 us - 22.301305860612082%
[Avg&Per] (C)get_pageHeader2436.5129999999995 us - 0.22878239411203669%2198.3668000000007 us - 0.27599779517870476%2319.9517000000005 us - 0.29041416798711567%3134.228800000001 us - 0.3649635223062446%
[Avg&Per] (D_1)decompress_pageData356587.5454999999 us - 33.482666568996265%31158.160799999983 us - 3.911805656191469%115629.7179 us - 14.474658381255693%29640.983400000016 us - 3.4515277590088287%
[Avg&Per] (D_2)decode_pageData457950.96670000016 us - 43.000434862259155%486085.0215000002 us - 61.02639204858503%485623.25309999986 us - 60.790866035802786%545723.8052999998 us - 63.54650374875476%
SUM1064991.4778 us796516.0731000002 us798842.4655999999 us858778.6472999998 us
[3] D_1 compare each step inside
[Avg&Per] (D-1)7_1_data_ByteBuffer_to_ByteArray(us)1269.7615999999996 us - 0.36377892330196115%1808.5776999999991 us - 7.207505165727991%1687.9916000000003 us - 1.412377327892232%3197.6313 us - 41.86667970732365%
[Avg&Per] (D-1)7_2_data_decompress_PageDataByteArray(us)345856.23130000004 us - 99.08569249502276%21247.39440000001 us - 84.67466169480036%116100.69619999993 us - 97.14384305311928%2432.4552000000012 us - 31.84820675254647%
[Avg&Per] (D-1)7_3_data_ByteArray_to_ByteBuffer(us)374.18470000000025 us - 0.10720162531460037%442.10720000000003 us - 1.761876157051776%424.26030000000026 us - 0.35498732863644405%421.1930000000002 us - 5.51469221169019%
[Avg&Per] (D-1)7_4_data_split_time_value_Buffer(us)1547.4220999999993 us - 0.443326956360674%1594.8989000000004 us - 6.355956982419887%1301.2614999999998 us - 1.0887922903520593%1586.372500000001 us - 20.770421328439692%
[3] D_2 compare each step inside
[Avg&Per] (D-2)8_1_createBatchData(us)3430.8672 us - 0.22855030377902266%4174.8958 us - 0.275155541260775%3438.1465 us - 0.22874736939899784%3730.6205 us - 0.24721094086707432%
[Avg&Per] (D-2)8_2_timeDecoder_hasNext(us)234016.77980000002 us - 15.589238228946503%236135.9462 us - 15.56305048087338%235278.4135 us - 15.65358490817499%234469.6457 us - 15.537217392727715%
[Avg&Per] (D-2)8_3_timeDecoder_readLong(us)357893.9434 us - 23.841426880277478%360253.1425 us - 23.743262865502565%358341.1524 us - 23.84121675993312%363063.0738 us - 24.058508247673558%
[Avg&Per] (D-2)8_4_valueDecoder_read(us)353821.6809 us - 23.570149451806063%359477.8899 us - 23.692168165422427%356440.4841 us - 23.714761161335133%355773.1939 us - 23.5754416723178%
[Avg&Per] (D-2)8_5_checkValueSatisfyOrNot(us)223758.3096 us - 14.905861011513531%224938.2721 us - 14.825043539994219%224282.1638 us - 14.921980483485841%226053.6587 us - 14.979528915812129%
[Avg&Per] (D-2)8_6_putIntoBatchData(us)328221.5562 us - 21.864774123677403%332305.597 us - 21.90131940694663%325251.7878 us - 21.639709317671908%325993.7043 us - 21.602092830601723%

...

编码方式GORILLAPLAINRLETS_2DIFF
dataset/disk/rl/zc_data/ZT11529.csv/disk/rl/zc_data/ZT11529.csv/disk/rl/zc_data/ZT11529.csv/disk/rl/zc_data/ZT11529.csv
pagePointNum(ppn)10000100001000010000
numOfPagesInChunk(pic)100100100100
chunksWritten(cw)13131313
timeEncoding(te)TS_2DIFFTS_2DIFFTS_2DIFFTS_2DIFF
valueDataType(vt)DOUBLEDOUBLEDOUBLEDOUBLE
valueEncoding(ve)GORILLAPLAINRLETS_2DIFF
compression(co)SNAPPYSNAPPYSNAPPYSNAPPY
totalPointNum12780287127802871278028712780287
tsfileSize(MB)23.1564121226.5996017522.1014785820.98550797
chunkDataSize_stats_mean(MB)1.8131843412.0811667441.7400725681.642661015
compressedPageSize_stats_mean(B)18948.6921758.6166718182.2991717160.8925
uncompressedPageSize_stats_mean(B)29684.757591463.4891725245.9341722210.36417
timeBufferSize_stats_mean(B)11461.462511461.462511461.462511461.4625
valueBufferSize_stats_mean(B)18221.268338000013782.44510746.875
[2] category: (A)get ChunkStatistic->(B)load on-disk Chunk->(C)get PageStatistics->(D)load in-memory PageData
[Avg&Per] (A)get_chunkMetadatas88753.03020000001 us - 11.21274000219672%97256.88870000001 us - 12.181751942467722%84509.5108 us - 9.498496964807579%86370.7961 us - 11.6687579647861%
[Avg&Per] (B)load_on_disk_chunk105215.9241 us - 13.29260306229143%138228.6108 us - 17.313597737139155%139816.5536 us - 15.714765088895252%96826.5282 us - 13.081335047881257%
[Avg&Per] (C)get_pageHeader2352.1583 us - 0.2971632563133493%2625.5312999999983 us - 0.328856613051254%3388.6254999999987 us - 0.38086658793698225%3159.255499999999 us - 0.4268177375109232%
[Avg&Per] (D_1)decompress_pageData114457.26139999992 us - 14.460120522641777%182862.35910000015 us - 22.904124612107367%155021.76800000004 us - 17.42376424722015%120684.50350000002 us - 16.304565026949902%
[Avg&Per] (D_2)decode_pageData480759.0216999998 us - 60.73737315655672%377408.39589999994 us - 47.27166909523449%506978.10549999995 us - 56.982107111140046%433147.3443000001 us - 58.51852422287182%
SUM791537.3956999998 us798381.7858000002 us889714.5634 us740188.4276000002
[3] D_1 compare each step inside
[Avg&Per] (D-1)7_1_data_ByteBuffer_to_ByteArray(us)1754.3673999999999 us - 1.6860113426598728%2320.3632000000002 us - 1.489795195953362%1825.1960999999988 us - 1.548613686329812%1607.3271999999997 us - 1.758246074195737%
[Avg&Per] (D-1)7_2_data_decompress_PageDataByteArray(us)100527.56879999992 us - 96.61067644486589%151532.73299999998 us - 97.29198327791244%113686.73430000001 us - 96.4591326329927%87212.82 us - 95.4016073295714%
[Avg&Per] (D-1)7_3_data_ByteArray_to_ByteBuffer(us)431.9958999999999 us - 0.41516388607230165%468.2119000000003 us - 0.30061666178303303%474.28859999999975 us - 0.4024169332984032%663.3730000000002 us - 0.7256599483773118%
[Avg&Per] (D-1)7_4_data_split_time_value_Buffer(us)1340.373799999999 us - 1.288148326401935%1429.1740000000002 us - 0.9176048643511714%1873.7816000000007 us - 1.5898367473790769%1932.989900000001 us - 2.1144866478555286%
[3] D_2 compare each step inside
[Avg&Per] (D-2)8_1_createBatchData(us)3348.9259 us - 0.2218271110091239%3442.3069 us - 0.24318767981262365%3466.6887 us - 0.22277034162258696%3338.6453 us - 0.22194982158403212%
[Avg&Per] (D-2)8_2_timeDecoder_hasNext(us)235317.741 us - 15.587043790733997%244839.8323 us - 17.29713023052909%239308.8755 us - 15.37805224577913%240223.005 us - 15.969786637750962%
[Avg&Per] (D-2)8_3_timeDecoder_readLong(us)361562.5392 us - 23.949282819264262%363364.4154 us - 25.67050285597614%362143.5815 us - 23.271443255687238%357833.1113 us - 23.788389623148674%
[Avg&Per] (D-2)8_4_valueDecoder_read(us)356325.4946 us - 23.602389962111484%241625.5526 us - 17.07005192367858%396438.5566 us - 25.475247513902037%335421.7093 us - 22.298501890735768%
[Avg&Per] (D-2)8_5_checkValueSatisfyOrNot(us)225554.2633 us - 14.940327764084076%232364.86419999998 us - 16.415814695306036%224510.63450000001 us - 14.4271133273284%230332.4743 us - 15.312273986066641%
[Avg&Per] (D-2)8_6_putIntoBatchData(us)327591.9399 us - 21.699128552797042%329856.9807 us - 23.30331261469754%330303.223 us - 21.225373315680617%337085.3345 us - 22.409098040713936%


补充实验

时间戳和值都使用PLAIN编码，然后改变压缩。

RLCompressionRealExpScripts.sh
压缩方式GZIPLZ4SNAPPYUNCOMPRESSED
dataset



pagePointNum(ppn)



numOfPagesInChunk(pic)



chunksWritten(cw)



timeEncoding(te)



valueDataType(vt)



valueEncoding(ve)



compression(co)



totalPointNum



tsfileSize(MB)



chunkDataSize_stats_mean(MB)



compressedPageSize_stats_mean(B)



uncompressedPageSize_stats_mean(B)



timeBufferSize_stats_mean(B)



valueBufferSize_stats_mean(B)



[2] category: (A)get ChunkStatistic->(B)load on-disk Chunk->(C)get PageStatistics->(D)load in-memory PageData
[Avg&Per] (A)get_chunkMetadatas



[Avg&Per] (B)load_on_disk_chunk



[Avg&Per] (C)get_pageHeader



[Avg&Per] (D_1)decompress_pageData



[Avg&Per] (D_2)decode_pageData



SUM



[3] D_1 compare each step inside
[Avg&Per] (D-1)7_1_data_ByteBuffer_to_ByteArray(us)



[Avg&Per] (D-1)7_2_data_decompress_PageDataByteArray(us)



[Avg&Per] (D-1)7_3_data_ByteArray_to_ByteBuffer(us)



[Avg&Per] (D-1)7_4_data_split_time_value_Buffer(us)



[3] D_2 compare each step inside
[Avg&Per] (D-2)8_1_createBatchData(us)



[Avg&Per] (D-2)8_2_timeDecoder_hasNext(us)



[Avg&Per] (D-2)8_3_timeDecoder_readLong(us)



[Avg&Per] (D-2)8_4_valueDecoder_read(us)



[Avg&Per] (D-2)8_5_checkValueSatisfyOrNot(us)



[Avg&Per] (D-2)8_6_putIntoBatchData(us)




人工数据实验结果

改变压缩方式

RLCompressionSynExpScripts.sh

...