|Authors||D. Pradhan, S. Wang, S. Ali, T. Yue and M. Liaaen|
|Title||Employing Rule Mining and Multi-Objective Search for Dynamic Test Case Prioritization|
|Project(s)||The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines , MBT4CPS: Model-Based Testing For Cyber-Physical Systems|
|Publication Type||Technical reports|
|Year of Publication||2018|
|Publisher||Simula Research Laboratory|
|Keywords||Black-box regression testing, Dynamic test case prioritization, Multi-objective optimization, Rule Mining, Search|
Test case prioritization (TP) is widely used in regression testing for optimal reordering of test cases to achieve specific criteria (e.g., higher fault detection capability) as early as possible. In our earlier work, we proposed an approach for black-box dynamic TP using rule mining and multi-objective search (named as REMAP) by defining two objectives (fault detection capability and test case reliance score) by considering test case execution results at runtime. In this paper, we conduct an extensive empirical evaluation of REMAP by employing three different rule mining algorithms and three different multi-objective search algorithms, and we also evaluate REMAP with one additional objective (estimated execution time) for a total of 18 different configurations of REMAP. Specifically, we empirically evaluated the 18 variants of REMAP with 1) two variants of random search while using two objectives and three objectives, 2) three variants of greedy algorithm based on one objective, two objectives, and three objectives, 3) 18 variants of static search-based prioritization approaches, and 4) six variants of rule-based prioritization approaches using two industrial and three open source case studies. Results showed that the two best variants of REMAP with two objectives and three objectives significantly outperformed the best variants of competing approaches by 84.4% and 88.9%, and managed to achieve on average 14.2% and 18.8% higher Average Percentage of Faults Detected per Cost (APFDc) scores.
This research was supported by the Research Council of Norway (RCN) funded Certus SFI (grant no. 203461/O30). Tao Yue and Shaukat Ali are also supported by RCN funded Zen-Configurator project (grant no. 240024/F20) and RCN funded MBT4CPS project.