Unsupervised Testing and Verification for Software Systems of Ground Autonomous Vehicles

Project begins 2023, estimated 3-year duration.

Army robotic ground vehicles are hardware-software systems. These systems combine hardware propulsion, actuation, sensing and other such capabilities with the decision-making, data storage, and other software driven operations. The Army has developed methods for reliability testing for its hardware (i.e. vehicle) systems. However, the need for verification and validation of the long-term software systems of ground robotic vehicles is identified by DEVCOM GVSC as one of the GVSC Robotic and Autonomy needs. The need for tools and capabilities that can be used for verification and validation of software is well recognized in the literature, particularly since half of the entire software development lifecycle cost originates from testing efforts. In addition retesting software after upgrades accounts for eighty percent of the entire maintenance cost

The objective is to develop a software testing and verification process that identifies diverse types of failure, searches the entire domain of input parameters to the software system that is tested, and does not need predefined ideal responses in order to determine faults (i.e. operates as an unsupervised process).

These research objectives will answer the following questions: Verify that the operation of the new fault metric based on the instance discrimination approach properly discovers faults. Is the new unsupervised process for testing for faults in software modules and systems of autonomy-focused applications equally capable with a process where the ideal behaviors are prescribed? How to demonstrate that diversity in the generated faults and in the input selections are captured by the new fitness function. How more efficient in discovering faults is a GA (genetic algorithm) driven search compared to a random search. How the new unsupervised testing capability operates for autonomy software of interest to GVSC.

The uniqueness of the proposed research stems from several factors: Developing an unsupervised instance discrimination method for determining improper software functionality (fault metric). Developing metrics that capture diversity both in the input selections that test the software and in the encountered fault conditions. Construct a fitness function that captures faults, input diversity and fault diversity. Combine a GA optimization method with the aforementioned fitness metrics for establishing the new unsupervised software testing and verification process. Demonstrate the new functionality on a system of interest to the sponsor.

#5.22