|Title||Testing Robotics Software using Constraint Programming in a Continuous Integration Process|
|Afilliation||Software Engineering, Software Engineering|
|Publication Type||PhD Thesis|
|Year of Publication||2015|
|Publisher||University of Stavanger|
|ISBN Number||ISBN 978-82-7644-613-5|
Testing complex integrated robots (CIRs) requires testing several interacting
control systems. This task is challenging, especially for robots
performing process-intensive tasks such as painting or gluing, since their
dedicated process control systems can be loosely coupled with the robot's
Current practices for validating CIRs involve manual test case design
and execution. To reduce testing costs and improve quality assurance, one
trend has been to automate the generation of test cases and execute the
test case automatically as part of a continuous integration process.
This thesis makes two main contributions. First, we present a methodology
for the fully automated testing of CIR control systems. Our approach
is based on a novel constraint-based model for automatically generating
test sequences, where test sequences are both generated and executed as
part of a continuous integration process. We call the methodology CATS,
which is short for Constraint-based Automatic Testing of IPS, where IPS
is an abbreviation for ABB Robotics' integrated paint control system.
Second, we present TC-Sched, a cost-eective method for automatic
test case execution scheduling on multiple machines with constraints on
accessible resources, such as measurement devices or network equipment.
TC-Sched is also based on a constraint-based model and is designed to be
integrated with and executed as part of continuous integration process.
The combinations of CATS and TC-Sched represent an ecient method
for quickly validating the critical software components of CIRs. We rst
use CATS to automatically generate test cases. We then use TC-Sched
to optimally schedule the test cases. We show that, when operating in a
continuous integration process, there is a trade-o between the time spent
solving the constraint model and the time spent executing the result of
To evaluate our approach, we integrated CATS within ABB Robotics'
continuous integration process. A full lab for the automatic testing of
actual embedded control systems was built. The rst version of the model
was introduced two years ago and has been extended several times. For
TC-Sched, the method was tested on several real industrial test suites, in
addition to a large set of randomly generated test suites. The results are
promising and ABB Robotics has decided to implement TC-Sched in a
full-scale setting at its research facilities at Bryne.
In conclusion, the research presented in this thesis shows that solving
constraint programming models as part of a continuous integration process
can generate realistic test cases and ecient test execution schedules. The
research also shows that the solving process can be performed so that the
trade-o between solving time and execution time is optimal.