• Buffered writes - IGNITE-21132
    • We need to write dump data in the buffers. Right now we write data directly to the disk.
      This leads to the suboptimal write performance.
      We must combine entries to the buffers to provide better performance.
  • Mostly sequential writes - 
    • Currently, each entry change during dump creation saved on the disk in corresponding dump file.
      This leads to the random writes.
      While modern SSD provides good performance for random writes, actual deployments can try to write dump to HDD or other block storages.
      We must enhance creation procedure to make writes as sequential as possible.
  • GridCacheVersion from primary (consistent cut adoption)
    • We use max  GridCacheVersion to effectively filter out changes that happens AFTER dump creation start on primary (because entry version assigned on primary).
    • To make dump creation effective on backups we can send max GridCacheVersion from primary to backups on dump start.
  • DumpConsumer that prints DumpEntry as json - to simplify data processing we must provide DumpConsumer that prints Ignite CacheEntry save in dump as json text.
    • It seems that GridRestProcessor can do BinaryObject → json transformation, already.
    • So during implementation we must unify the way we do BinaryObject → json.
    • We can add print as json to control.sh --cache scan command, also.
  • Revise dump structure - to make write fully sequential we can try to create dump as one file (or one file per cache group).
    • This will change a bit DumpReader API but combined with buffered writes can provide an optimal performance.
  • All improvements must be based on proper performance testing.
  • No labels

Apache Ignite