Versions Compared

Old Version 3

changes.mady.by.user guozhang Wang

Saved on

compared with

New Version 4

changes.mady.by.user guozhang Wang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

We can still let advanced users to specify different configs for 1) block cache size, 2) Memtable buffer size, 3) number of Memtables, 4) etc for a single KafkaStreams instance, and be aware that the amount of memory for caching will be the total specified amount of memory minus the total amount of memory used by these state stores (thus, they need to be aware of the equation to calculate the TOTAL amount of memory used based on these config values); and when users do not specify these configs we should dynamically set some default values based on the total specified amount of memory to make sure we still have some amount left for caching.

  

And as a side note, if we are using bytes in our own caching layer as proposed above, then we should try to reduce the usage of RocksDB's own Memtable as it effectively have less benefits additionally.