Pull Request review bot

In order to improve the round-trip times as well as the user experience of filing Pull Requests against MXNet, this process could be monitored and assisted by a bot which takes care of finding appropriate reviewers, reminding them and also generating statistics to find areas of improvement.

Link to dev List discussion

TODO

Feature Shepherd

TODO

Problem

The MXNet community faces a high round-trip due to the big volume of Pull Requests paired with the lack of reviewers. This creates frustration on the contributors' side since they get no or late feedback in a majority of the cases. The lack of reviewers is partially caused by the fact that people are not aware of Pull Requests that are in their area of expertise and non-committers unable to use the CODEOWNERS feature to be notified. Frequent reviewers on the other hand don't get rewarded or incentivised for their activity since there's no active metric that tracks these endeavours.

Proposed Approach

We divide the lifecycle of a Pull Request into multiple stages which would generally look as follows:

Work in progress
Triage
Subject matter expert review
[Committer review]
Merged

Whenever a new Pull Request gets created, it gets set to "Work in progress" (WIP). As soon as the contributor considers their PR final and ready for review, they issue a command to move it to the triage stage. This stage is like the "first level support" which addresses general topics. After the initial review is done, the subject matter expert is engaged to give a more thorough review. In case the subject matter expert was not a committer, an additional committer review gets requested in order to get them to merge the PR after their review.

Whenever a review was given, the PR moves into the "Requested changes" state to signal the contributor that there is outstanding feedback as well as to show the reviewers that there's no action required from their side. As soon as the feedback has been processed, the contributor can issue a command to move it back into the "Ready for review" state.

The bot determines who to ask for review in every stage through a config-file in the repository. In there, people can add themselves in the same style as the CODEOWNERS file to

Whenever a new Pull Request gets created, the bot analyses the changed files in the same style as the CODEOWNERS feature. By having a configuration file in the MXNet repository, we can assign certain folders and files to certain people. The configuration would allow the following variants of responsibility:

MXNet Java Inference API#ProposedApproach

MXNet Java Inference API#ClassDiagram

MXNet Java Inference API#SequenceDiagram