Versions Compared

Old Version 43

changes.mady.by.user Maxim Muzafarov

Saved on

compared with

New Version 44

changes.mady.by.user Maxim Muzafarov

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

CPU(s): 56
CPU Model name: Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
SSD: HPE 960GB SATA 6G (Reads 530 MiB/s , Writes 490 MiB/s)
HDD: HP 9200 20TB SAS
Network I\O: 10Gb/s full duplex bandwidth

Cluster configuration

The Apache Ignite cluster configured to be consisting of two nodes, replicated cache configured with 1024 partitions and filled with dummy entries total size of 78 Gb.
The example configuration used for profiling rebalance procedure with persistence enabled – example-rebalance.xml. 

Profiling java code

Persistence enabled

SSD


Code Block
batches            : 146938   
rows               : 77321844 
rows per batch     : 526      

time (total)       : 20 min   
cache size         : 78055 MB 
rebalacne speed    : 63 MB\sec
rows per sec       : 62965 rows
batch per sec      : 119 batches


+ cache rebalance total                          : 1228051 ms : 100.00
+ + preload on demander                          : 1175260 ms : 95.70 
+ + + offheap().invoke(..)                       : 936040 ms  : 76.22 
+ + + + dataTree.invoke(..)                      : 895937 ms  : 72.96 
+ + + + + BPlusTree.invokeDown(..)               : 147120 ms  : 11.98 <<-!
+ + + + + FreeList.insertDataRow(..)             : 707925 ms  : 57.65 <<-!
+ + + + CacheDataStoreImpl.finishUpdate(..)      : 9154 ms    : 0.75  
+ + + ttl().addTrackedEntry(..)                  : 8149 ms    : 0.66  
+ + + wal().log(..)                              : 138571 ms  : 11.28 
+ + + continuousQueries().onEntryUpdated(..)     : 7287 ms    : 0.59  
+ message serialization                          : 1617 ms    : 0.13  
+ network delay between meesages (total)         : 15129 ms   : 1.23  
+ make batch on supplier handleDemandMessage(..) : 240618 ms  : 19.85

HDD


Code Block
batches            : 146938   
rows               : 77321844 
rows per batch     : 526      

time (total)       : 40 min   
cache size         : 78055 MB 
rebalacne speed    : 31 MB\sec
rows per sec       : 31470 rows

+ cache rebalance total                          : 2456973 ms : 100.00
+ + preload on demander                          : 2415154 ms : 98.30 
+ + + offheap().invoke(..)                       : 1640175 ms : 66.76 
+ + + + dataTree.invoke(..)                      : 1595260 ms : 64.93 
+ + + + + BPlusTree.invokeDown(..)               : 220390 ms  : 8.97  
+ + + + + FreeList.insertDataRow(..)             : 1340636 ms : 54.56 <<-!
+ + + + CacheDataStoreImpl.finishUpdate(..)      : 10807 ms   : 0.44  
+ + + ttl().addTrackedEntry(..)                  : 9678 ms    : 0.39  
+ + + wal().log(..)                              : 664680 ms  : 27.05 <<-!
+ + + continuousQueries().onEntryUpdated(..)     : 8521 ms    : 0.35  
+ message serialization                          : 1618 ms    : 0.07  
+ network delay between nodes                    : 7788 ms    : 0.32  
+ make batch on supplier handleDemandMessage(..) : 185749 ms  : 7.59


In-memory only


Code Block
batches            : 150701    
rows               : 79355844  
rows per batch     : 526       

time (total)       : 5.5 min   
cache size         : 79852 MB  
rebalacne speed    : 234 MB\sec
rows per sec       : 232715 rows

+ cache rebalance total                          : 341524 ms : 100.00
+ + preload on demander                          : 306950 ms : 89.88 
+ + + offheap().invoke(..)                       : 228015 ms : 66.76 
+ + + + dataTree.invoke(..)                      : 195239 ms : 57.17 
+ + + + + BPlusTree.invokeDown(..)               : 71207 ms  : 20.85 <<-!
+ + + + + FreeList.insertDataRow(..)             : 121611 ms : 35.61 <<-!
+ + + + CacheDataStoreImpl.finishUpdate(..)      : 9988 ms   : 2.92  
+ + + ttl().addTrackedEntry(..)                  : 10523 ms  : 3.08  
+ + + continuousQueries().onEntryUpdated(..)     : 9665 ms   : 2.83  
+ message serialization                          : 1307 ms   : 0.38  
+ network delay between nodes                    : 23409 ms  : 6.85  
+ make batch on supplier handleDemandMessage(..) : 90102 ms  : 26.39

...

REPLICATED Cache size (total): 63121 MB 83839 MB
Rebalance time (total): 1343130 ms 1421295 ms
Rebalance speed (avg): 47 59 MB/s


CPU utilization


CPU user time

...