Label Prediction by the @mxnet-label-bot

Description:

Currently, within the incubator-mxnet repo there are over 800+ issues and new ones being generated every day. The goal is to ease this process and handle developer's issues in an appropriate manner. With the use of labelling, experts in their respective areas can provide the help that users face. We employ the label bot today to help ease the issue/pull request process. Given the data which the repository provides of issues and pull requests which have been previously labelled, an interesting use case of this data opens up. Based upon the data of this repository, we are able to provide insights and predictions of labels on new issues and pull requests. This mechanism will provide a better experience for those who have raised an issue to get a faster response, and it allows for existing and new contributors to better filter for their areas of expertise who are wanting to help out welcoming new users.

Background:

Note: Training data here is limited (~13,187 issues) but after the data cleaning process we expect this value to be greatly further reduced.

Metrics

Multi-label Classification:
Accurate prediction of at least one label in an issue: ~87%
Accuracy in predicting all labels in an issue (may not be able to get all labels necessarily in an issue): ~20%

(Last collect issue id — 13193. Date: 11/8)
(These were labels chosen for prediction initially by the model — issues specific to these labels are what is tested on)


Results in accurately predicting a label (The model predicted a label and that was one of the actual labels in the repo):

Classification Accuracy: