DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
Status
Current state: Under DiscussionVoting
Discussion thread: here
Vote Thread: here
JIRA: here
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
...
Metric Semantics and Behaviour
| Scenario | rchar | wchar | syscr | syscw | cancelled_write_bytes |
|---|---|---|---|---|---|
Normal steady-state operation | Available (MB-GB range) | Available (MB-GB range) | Available (thousands-millions) | Available (thousands-millions) | Available (MB range low) |
Broker cold start (empty cache) | Equals read_bytes initially | Available | Available | Available | Minimal (near zero) |
High cache hit ratio (>90%) | Much larger than read_bytes | Available | Available | Available | Available |
Low cache hit ratio (<20%) | Close to read_bytes value | Available | Available | Available | Available |
Write amplification scenario | Available | Much smaller than write_bytes | Available | Available | Available |
Optimal I/O batching | Available | Available | Low count (large avg size) | Low count (large avg size) | Available |
Poor I/O batching (tiny reads/writes) | Available | Available | Very high count (small avg size) | Very high count (small avg size) | Available |
Active log compaction | Available | Available | Available | Available | High (MB-GB range) |
No log compaction activity | Available | Available | Available | Available | Low (near zero) |
Topic deletion in progress | Available | Available | Available | Available | Elevated (MB range) |
Non-Linux platform | N/A | N/A | N/A | N/A | N/A |
Linux platform without /proc/self/io | N/A | N/A | N/A | N/A | N/A |
Metric Cardinality:
- Question: How many metric instances will this create per broker?
- The proposed I/O metrics have low cardinality - specifically 7 metrics per broker process.
- These metrics are process-level counters, not topic-level (like `BytesInPerSec` (per topic), partition-level ( `LogEndOffset`), or replica-level metrics(`UnderReplicatedPartitions`).
- Cardinality Breakdown - Per Broker Process:
- `linux-disk-read-bytes`: 1 metric (existing)
- `linux-disk-write-bytes`: 1 metric (existing)
- `linux-disk-rchar`: 1 metric (new)
- `linux-disk-wchar`: 1 metric (new)
- `linux-disk-syscr`: 1 metric (new)
- `linux-disk-syscw`: 1 metric (new)
- `linux-disk-cancelled-write-bytes`: 1 metric (new)
...