Page History

...

the new volumes streamed the data from the other brokers generating a lot of new data to be written to disk, this + compacitons + produce requests reached the hard limit of the dirty pages forcing the OS to start the sync writes and drastically degrading the produce requests.

As far as the new broker is copying partitions they become again the leaders and start getting produce requests, but the broker is still under pressure because of the data being streamed,

In our case we were able to address this issue playing a bit with the OS resources and the OS dirty pages configs, but it would have been great if we had a metric to monitor when the produce requests get close to become "sync"

...