AuthorsM. Zhang, S. Ali, T. Yue and Malin Hedman
TitleUncertainty-based Test Case Generation and Minimization for Cyber-Physical Systems: A Multi-Objective Search-based Approach
AfilliationSoftware Engineering
Project(s)U-Test: Testing Cyber-Physical Systems under Uncertainty: Systematic, Extensible, and Configurable Model-based and Search-based Testing Methodologies, The Certus Centre (SFI)
Publication TypeTechnical reports
Year of Publication2017
PublisherSimula Research Laboratory
KeywordsCyber-Physical Systems, Multi-Objective Search, Test Case Generation, Test Case Minimization, Uncertainty

Cyber-Physical Systems (CPSs) typically operate in highly indeterminate environmental conditions, which require the development of testing methods that must explicitly consider uncertainty in test design, test generation, and test optimization. Towards this direction, we propose uncertainty-wise test case generation and test case minimization strategies that rely on test ready models explicitly specifying subjective uncertainty. We propose two test case generation strategies and four test case minimization strategies based on Uncertainty Theory and multi-objective search. These strategies include a novel methodology for designing and introducing indeterminacy sources in the environment during test execution and a novel set of uncertainty-wise test verdicts. We performed an extensive empirical study to select the best algorithm out of eight commonly used multi-objective search algorithms, for each of the four minimization strategies, with five use cases of two industrial CPS case studies. The minimized set of test cases obtained with the best algorithm for each minimization strategy were executed on the two real CPSs. The results showed that our best test strategy managed to observe 51% more uncertainties due to unknown indeterminate behaviors of the physical environment of the CPSs as compared to the rest of the test strategies. In addition, the same test strategy managed to observed 118% more unknown uncertainties as compared to the unique number of known uncertainties


This work is supported by U-Test project (Testing Cyber-Physical Systems under Uncertainty).

Citation Key24840