AuthorsS. Ali, Y. Li, T. Yue and M. Zhang
TitleAn Empirical Evaluation of Mutation and Crossover Operators for Multi-Objective Uncertainty-Wise Test Minimization
AfilliationSoftware Engineering
Project(s)U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI)
Publication TypeTechnical reports
Year of Publication2017
PublisherSimula Research Laboratory
KeywordsCyber-Physical Systems, Multi-Objective Search, Test Case Minimization, Uncertainty-Wise Testing

Multi-objective uncertainty-wise test case minimization focuses on selecting a minimum number of test cases to execute out of all available ones while maximizing effectiveness (e.g., coverage), minimizing cost (e.g., time to execute test cases), and at the same time optimizing uncertainty-related objectives. In our previous work, we developed four uncertainty-wise test case minimization strategies relying on Uncertainty Theory and multi-objective search (NSGA-II with default settings), which were evaluated with one real Cyber-Physical System (CPS) with inherent uncertainty. However, a fundamental question to answer is whether these default settings of NSGA-II are good enough to provide optimized solutions. In this direction, we report one of the preliminary empirical evaluations, where we performed an experiment with three different mutation operators and three crossover operators, i.e., in total nine combinations with NSGA-II for the four uncertainty-wise test case minimization strategies using a real CPS case study. Results show that the Blend Alpha crossover operator together with the polynomial mutation operator permits NSGA-II achieving the best performance in terms of solving our uncertainty-wise test minimization problems.

Citation Key25014