Skip to main content
arc logo
Automotive Research Center
hero image
Back to all projects

Vehicle Controls & Behaviors

Annual Plan

Evaluating Sensitivity of Autonomous Algorithms to Sensor Error and Environmental Conditions

Project Team

Principal Investigator

Daniel Carruth, Mississippi State University


Paramsothy Jayakumar, U.S. Army GVSC


Chris Goodin, Lalitha Dabbiru, Mississippi State University


Nick Gaul, RAMDO Solutions


Nick Scherer, Mississippi State University

Project Summary

Project began April 2019.

While much effort has been devoted to the development of autonomous navigation algorithms in recent years, techniques and procedures for quantifying the performance of these algorithms have been developed on an ad-hoc, case-by-case basis. In particular, the use of system-level tests to compare the performance of autonomy subsystems (such as path planning or navigation) leads to a high degree of uncertainty in algorithmic performance. In this work, we propose to develop a numerical model for quantifying the propagation of error through the subsystems of autonomous navigation software.

The performance of autonomous ground systems has typically been measured at the system level, and subsystem-level information is often obscured to the tester. An analog in traditional manned-vehicle testing might be developing a model of powertrain influence on maximum speed. For example, the maximum speed of a vehicle depends on many factors other than the powertrain (aerodynamics of the vehicle, tire shape and inflation, etc.). If we wish to isolate the influence of the powertrain, we must isolate the powertrain in a system-level test such as a dynamometer. Similarly, in an autonomous vehicle, if we would like to understand the performance of the path planning algorithm, we must devise a test to isolate this algorithm (from perception, global planning, etc.) at a system level. Therefore, the objective of this research is to study test methods for autonomous vehicles in simulation to determine how to design tests that isolate each component of the autonomous architecture.


  • C Goodin, D Carruth, L Dabbiru, N Scherer., “Predicting Error Propagation in Autonomous Systems,” 2020 NDIA Ground Vehicle Systems Engineering and Technology, November 2020.
  • Dabbiru, L., Goodin, C., Scherrer, N., and Carruth, D., “LiDAR Data Segmentation in Off-Road Environment Using Convolutional Neural Networks (CNN),” SAE Technical Paper 2020-01-0696, 2020,

Recent Publications from Prior Related Work:

  • Hudson, C. R., Goodin, C., Doude, M., & Carruth, D. W. (2018, August). Analysis of Dual LIDAR Placement for Off-Road Autonomy Using MAVS. In 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA) (pp. 137-142). IEEE..
  • Goodin, C., Doude, M., Hudson, C., & Carruth, D. (2018). Enabling off-road autonomous navigation simulation of LIDAR in dense vegetation. Electronics, 7(9), 154.
  • Durst, P. J., Goodin, C. T., Bethel, C. L., Anderson, D. T., Carruth, D. W., & Lim, H. (2018). A Perception- Based Fuzzy Route Planning Algorithm for Autonomous Unmanned Ground Vehicles. Unmanned Systems, 6(04), 251-266.
  • Goodin, C., Carruth, D., Doude, M., Hudson, C., Dabbiru, L., & Sharma, S. (2019). Training of Neural Networks with Automated Labeling of Simulated Sensor Data (No. 2019-01-0120). SAE Technical Paper.
  • Goodin, C., Carrillo, J. T., McInnis, D. P., Cummins, C. L., Durst, P. J., Gates, B. Q., & Newell, B. S. (2017, May). Unmanned ground vehicle simulation with the Virtual Autonomous Navigation Environment. In Military Technologies (ICMT), 2017 International Conference on (pp. 160-165). IEEE.
  • Durst, P. J., Monroe, G., Bethel, C. L., Anderson, D. T., & Carruth, D. W. (2018, May). A history and overview of mobility modeling for autonomous unmanned ground vehicles. In Autonomous Systems: Sensors, Vehicles, Security, and the Internet of Everything (Vol. 10643, p. 106430G). International Society for Optics and Photonics.
  • Carruth, D. W. (2018, August). Simulation for Training and Testing Intelligent Systems. In 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA) (pp. 101-106). IEEE.
  • Carruth, D. W., & Bethel, C. L. (2017, January). Challenges with the integration of robotics into tactical team operations. In Applied Machine Intelligence and Informatics (SAMI), 2017 IEEE 15th International Symposium on (pp. 000027-000032). IEEE.