Deep Reinforcement Learning Approach to CPS Vehicle Re‐envisioning

Project Summary

Principal Investigators

Venkat N. Krovi (PI), Clemson University
Melissa Smith, Umesh Vaidya, Phanindra Tallapragada, Feng Luo. Rahul Rao (Co‐PIs), Clemson University

Students

Fei Ding, Max Faykus, Ameya Salvi, Jake Buzhardt, Alex Krolicki, Ajinkya Joglekar, Dan Zhang, Clemson University

Government

David Gorsich, Jon Smereka, Mark Brudnak, U.S. Army GVSC

Industry

Karthik Krishnan, MSC Software/Hexagon

Project #1.A75 began June 2020 and was completed Q4 2022.

The Army’s plans for Next‐Gen Ground Vehicles, for reconnaissance, combat as well as logistics, seek to re‐envision functionality in new classes of Cyber‐Physical System (CPS) vehicle concepts. Modern day CPS Systems offer opportunities to extract superior performance (reconfigurability, robustness, reliability) from the underlying: (i) articulated‐wheeled electromechanical vehicle platforms (e.g. GroundX vehicles with embodied‐mechatronic preflexes); (ii) multi‐modal multi‐resolution sensor‐based spatio‐temporal information gathering; coupled closely with (iii) size, weight and power (SWaP) constrained algorithmic‐ intelligence to realize “real‐time Sense‐Think‐Act”. This inherent but unexploited capability offers enormous opportunity for superior mobility and information gathering in a range of outdoor terrains.

Intelligent orchestration/coordination of newly provisioned flexibility, from the myriad parameters, settings and behaviors, requires careful evaluation at both design stage and during operations. Our team developed a Deep Reinforcement Learning Enhanced Decision‐Support Framework for CPS System Evaluation enabled by multi‐domain physical and sensor‐intelligence modeling of the ground vehicle. The framework and methods will be applicable across: (i) varied size/weight/architecture CPS vehicles, sensor‐allocation schema, sensor‐intelligence modules, and/or motion planner‐controllers; and (ii) diverse benchmark test‐environments/scenarios. In latter stages of the project the framework will enable simulation‐based design‐refinement and hardware‐in‐the‐loop testing.

Other Publication:

Salvi, A., Buzhardt, J., Tallapragada, P., Krovi, V., Brudnak, M., and J. M. Smereka, “Deep reinforcement learning for simultaneous path planning and stabilization of offroad vehicles”, In Proceedings of the Ground Vehicle Systems Engineering and Technology Symposium (GVSETS), NDIA, Novi, MI, Aug. 10-12, 2021.”

1.A75

Publications:

Salvi, A., Buzhardt, J., Tallapragda, P., Krovi, V. N., Smereka, J. M., & Brudnak, M. (2022). Virtual Evaluation of Deep Learning Techniques for Vision-Based Trajectory Tracking. SAE International Journal of Advances and Current Practices in Mobility, 5(2022-01-0369), 326-334.
Joglekar, A., Krovi, V., Brudnak, M., & Smereka, J. M. (2022, June). Hybrid Reinforcement Learning based controller for autonomous navigation. In 2022 IEEE 95th Vehicular Technology Conference:(VTC2022-Spring) (pp. 1-6). IEEE.
Buzhardt, J., & Tallapragada, P. (2022). A Koopman operator approach for the vertical stabilization of an off-road vehicle. IFAC-PapersOnLine, 55(37), 675-680.
Salvi, A., Coleman, J., Buzhardt, J., Krovi, V., & Tallapragada, P. (2022). Stabilization of vertical motion of a vehicle on bumpy terrain using deep reinforcement learning. IFAC-PapersOnLine, 55(37), 276-281.
Krolicki, A., Tellez-Castro, D., & Vaidya, U. (2022, December). Nonlinear dual-mode model predictive control using Koopman eigenfunctions. In 2022 IEEE 61st Conference on Decision and Control (CDC) (pp. 3074-3079). IEEE.
Ding, F., Yang, Y., Hu, H., Krovi, V., & Luo, F. (2022). Dual-level knowledge distillation via knowledge alignment and correlation. IEEE Transactions on Neural Networks and Learning Systems, 35(2), 2425-2435.