...

The following metrics will be exposed in the Kafka Streams' metrics

• num-immutable-mem-table
• mem-table-flush-pending
• compaction-pending
• background-errors
• cur-size-active-mem-table
• cur-size-all-mem-tables
• size-all-mem-tables
• num-entries-active-mem-table
• num-entries-imm-mem-tables
• num-deletes-active-mem-table
• num-deletes-imm-mem-tables
• estimate-num-keys
• estimate-table-readers-mem
• mem-table-flush-pending
• num-running-flushes
• compaction-pending
• num-running-compactions
• estimate-pending-compaction-bytesnum-live-versions
• total-sst-files-size
• live-sst-files-size
• estimate-pending-compaction-bytes
• num-runninglive-compactions
• num-running-flushes
• estimate-oldest-key-time
• versions
• block-cache-capacity
• block-cache-usage
• block-cache-pinned-usage
• estimate-num-keys
• estimate-table-readers-mem
• estimate-oldest-key-time
• background-errors

The recording level for all metrics will be INFO.

...

Total number of entries in the active memtable. sum for For segmented state stores, the sum of number of entries over all segments is reported.

num-entries-imm-mem-tables

Total number of entries in the unflushed immutable memtables. sum for For segmented state stores, the sum of number of entries over all segments is reported.

num-deletes-active-mem-table

Total number of delete entries in the active memtable. sum for For segmented state stores, the sum of number of deletes over all segments is reported.

num-deletes-imm-mem-tables

Total number of delete entries in the unflushed immutable memtables. sum for For segmented state stores, the sum of number of deletes over all segments is reported.

mem-table-flush-pending

This metric returns 1 if a memtable flush is pending; otherwhise it returns 0. sum for For segmented state stores, the sum of pending flushes over all segments is reported.

num-running-flushes

Number of currently running flushes. sum for segmented storesFor segmented state stores, the sum of running flushes over all segments is reported.

compaction-pending

This metric 1 if at least one compaction is pending; otherwise, the metric reports 0. For segmented state stores, the sum of ones and zeros over all segments is reported.

num-running-compactions

Number of currently running compactions. For segmented state stores, the sum of the number of currently running compactions over all segments is reported.

...

Estimated total number of bytes a compaction needs to rewrite to get all levels down to under target size. This metric is not valid for compactions other than level-based. For segmented state stores, the sum of the estimated total number of bytes over all segments is reported.

...

This metric 1 if at least one compaction is pending; otherwise, the metric reports 0. For segmented state stores, the sum of ones and zeros over all segments is reported.

total-sst-files-size

Total size in bytes of all SST files. For segmented state stores, the sum of the sizes of SST files over all segments is reported.

...

block-cache-capacity

Block cache capacity. sum for For segmented state stores, the sum of the cache capacity over all segments is reported, if separate caches are used, otherwise, if only one cache is used, the cache capacity of any segment is reported.

block-cache-usage

Memory size for the entries residing in block cache. sum for For segmented state stores, the sum of the cache capacity over all segments is reported, if separate caches are used, otherwise, if only one cache is used, the cache capacity of any segment is reported.

block-cache-pinned-usage

Memory size for the entries being pinned. sum for For segmented state stores, the sum of the cache capacity over all segments is reported, if separate caches are used, otherwise, if only one cache is used, the cache capacity of any segment is reported.

estimate-num-keys

Estimated number of total keys in the active and unflushed immutable memtables and storage. For segmented state stores, the sum of the estimated number of keys over all segments is reported.

estimate-table-readers-mem

Estimated memory in bytes used for reading SST tables, excluding memory used in block cache (e.g., filter and index blocks). sum for For segmented state stores, the sum of the estimated memory over all segments is reported.

estimate-oldest-key-time

Estimated oldest key timestamp in the RocksDB instance. This metric is currently only available for FIFO compaction with compaction_options_fifo.allow_compaction = false. For segmented state stores, the minimum of the estimated oldest key timestamp over all segments is reported.

...

Accumulated number of background errors. For segmented state stores, the sum of the number of background errors over all segments is reported.

memtables-size

Memory usage of all mem-tables. Mem-tables are in-memory write buffers. Each new key-value pair is first written to a mem-table and each read looks first into the mem-table before it looks on disk. Once a mem-table is full it becomes immutable and it is replaced by a new mem-table. A background thread flushes a mem-table asynchronously to disk. Additionally, mem-tables can also be flushed manually. RocksDB keeps in memory the currently active mem-tables, full but not yet flushed mem-tables, and flushed mem-tables that are kept around to maintain write history in memory.

cache-size

Memory usage by caches. RocksDB caches data in memory for reads. By default, those caches contain only data blocks, i.e., uncompressed sequences of key-value pairs in sorted order. Therefore this cache is often referred to as block cache. However, users can configure RocksDB to also store index and filter blocks in the cache.

table-readers-memory-size

Memory usage of all the table readers. This is the memory RocksDB uses for reading SST tables, excluding the memory used in caches. This are for example index and filter blocks when they are not configured to be stored in the caches. 

Compatibility, Deprecation, and Migration Plan

...