|Authors||T. Rolfsnes, L. Moonen and D. Binkley|
|Editors||G. Ruso, M. Di Penta and T. N. Nguyen|
|Title||Predicting Relevance of Change Recommendations|
|Project(s)||evolveIT: Evidence-Based Recommendations to Guide the Evolution of Component-Based Product Families, The Certus Centre (SFI)|
|Publication Type||Proceedings, refereed|
|Year of Publication||2017|
|Conference Name||The IEEE/ACM International Conference on Automated Software Engineering (ASE)|
|Keywords||evolutionary coupling, random forests, recommendation confidence, targeted association rule mining|
Software change recommendation seeks to suggest artifacts (e.g., files or methods) that are related to changes made by a developer, and thus identifies possible omissions or next steps. While one obvious challenge for recommender systems is to produce accurate recommendations, a complimentary challenge is to rank recommendations based on their relevance. In this paper, we address this challenge for recommendation systems that are based on evolutionary coupling. Such systems use targeted association-rule mining to identify relevant patterns in a software system’s change history. Traditionally, this process involves ranking artifacts using interestingness measures such as confidence and support. However, these measures often fall short when used to assess recommendation relevance.
We propose the use of random forest classification models to assess recommendation relevance. This approach improves on past use of various interestingness measures by learning from previous change recommendations. We empirically evaluate our approach on fourteen open source systems and two systems from our industry partners. Furthermore, we consider complimenting two mining algorithms: CO-CHANGE and TARMAQ. The results find that random forest classification significantly outperforms previous approaches, receives lower Brier scores, and has superior trade-off between precision and recall. The results are consistent across software system and mining algorithm.