Evaluating Sensitivity of Autonomous Algorithms to Sensor Error and Environmental Conditions

Project began April 2019 and is to be completed early 2023.

While much effort has been devoted to the development of autonomous navigation algorithms in recent years, techniques and procedures for quantifying the performance of these algorithms have been developed on an ad-hoc, case-by-case basis. In particular, the use of system-level tests to compare the performance of autonomy subsystems (such as path planning or navigation) leads to a high degree of uncertainty in algorithmic performance. In this work, we propose to develop a numerical model for quantifying the propagation of error through the subsystems of autonomous navigation software.

The performance of autonomous ground systems has typically been measured at the system level, and subsystem-level information is often obscured to the tester. An analog in traditional manned-vehicle testing might be developing a model of powertrain influence on maximum speed. For example, the maximum speed of a vehicle depends on many factors other than the powertrain (aerodynamics of the vehicle, tire shape and inflation, etc.). If we wish to isolate the influence of the powertrain, we must isolate the powertrain in a system-level test such as a dynamometer. Similarly, in an autonomous vehicle, if we would like to understand the performance of the path planning algorithm, we must devise a test to isolate this algorithm (from perception, global planning, etc.) at a system level. Therefore, the objective of this research is to study test methods for autonomous vehicles in simulation to determine how to design tests that isolate each component of the autonomous architecture.

Publications:

• C Goodin, D Carruth, L Dabbiru, N Scherer., “Predicting Error Propagation in Autonomous Systems,” 2020 NDIA Ground Vehicle Systems Engineering and Technology, November 2020.
• Dabbiru, L., Goodin, C., Scherrer, N., and Carruth, D., “LiDAR Data Segmentation in Off-Road Environment Using Convolutional Neural Networks (CNN),” SAE Technical Paper 2020-01-0696, 2020, https://doi.org/10.4271/2020-01-0696.
• C Goodin, DW Carruth, L Dabbiru, CH Hudson, LD Cagle, N Scherrer, MN Moore, P Jayakumar, Simulation-based testing of Autonomous Ground Vehicles, SPIE Defense and Commercial Sensing 2022, Orlando, FL
• C Goodin, MN Moore, DW Carruth, CH Hudson, LD Cagle, J Paramsothy, An Empirical Vehicle Speed Model for Tuning Throttle Controller Parameters – in review, IEEE Transactions on Vehicular Technology
• D Carruth, C Goodin, L Dabbiru, N Scherer, M Moore, C Hudson, L Cagle, and P Jayakumar, “Measuring the Influence of Environmental Factors on Obstacle Detection and Avoidance with an Autonomous Ground Vehicle,” (under review).

#1.31