Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The accumulation of "received number of records" over a long period of time can hide a recent data skew event. The same can also hide a recent fix to an existing data skew problem. Therefore the proposed metric will need to look at the change in the received number of records within a recent period, similar to the existing "Backpressure" or "Busy" metrics on the Flink Job Graph, and show a "live" data skew score.

Additional Tab to List All Operators and Their Data Skew Score in Descending Order of Their Data Skew Score

...