Page History

This is a follow up task for the:Consumer threading refactor design

Objective

To ensure that our new Kafka consumer is robust, performant, and scalable by rigorously testing its capabilities in various scenarios, including different message sizes, numbers, consumer counts, and CPU throttling conditions.

...

1. Varying Message Sizes: Measure the rate of message consumption across different message sizes.

   • Metrics: Throughput (messages/sec), Latency

   • Tools: Kafka built-in monitoring, custom logging

2. Varying Message Numbers: Measure how well the consumer handles varying amounts of messages.

   • Metrics: Throughput, Backlog drain time

   • Tools: Kafka monitoring, custom logging

Number of Consumers

1. Single Consumer: Measure performance and resource utilization with a single consumer.

   • Metrics: Throughput, CPU, and Memory Usage

   • Tools: Kafka monitoring, top, vmstat

2. Multiple Consumers: Evaluate how the consumer scales horizontally.

   • Metrics: Aggregate Throughput, Per-Consumer Throughput, CPU , and Memory Usage

   • Tools: Kafka monitoring, top, vmstat

...

1. • Throttling

1. No Throttling: Baseline performance metrics.

   • Metrics: Throughput, CPU, and Memory UsageTools: Kafka monitoring, top, vmstat

2. 50% and 75% Throttling: Simulate CPU constraint scenarios.

   • Metrics: Throughput, Latency, CPU and Memory UsageTools: cgroups, Kafka monitoring

Special Scenarios

1. High Deserialization CPU Cost: Simulate a high-CPU cost deserialization algorithm.

   • Metrics: Throughput, Latency, CPU Utilization

   • Tools: Kafka monitoring, Profiling tools

2. Schema Registry: Measure the impact of using a schema registry for deserialization.

   • Metrics: Throughput, Latency, Schema registry lookup time

   • Tools: Kafka monitoring, Schema Registry logs