Deep Reinforcement Learning Approach to CPS Vehicle Re‐envisioning

Principal Investigators

• Venkat N. Krovi (PI), Clemson University
• Melissa Smith, Umesh Vaidya, Phanindra Tallapragada, Feng Luo. Raji; Rao (Co‐PIs), Clemson University

Faculty

• Yiqiang Han, Clemson University

Students

• Fei Ding, Max Faykus, Ameya Salvi, Jake Buzhardt, Alex Krolicki, Ajinkya Joglekar, Dan Zhang, Clemson University

Government

• Denise Rizzo, David Gorsich, Jon Smereka, Mark Brudnak, U.S. Army GVSC

Industry

• Karthik Krishnan, MSC Software/Hexagon

Project began June 2020 and was completed Q4 2022.

The Army’s plans for Next‐Gen Ground Vehicles, for reconnaissance, combat as well as logistics, seek to re‐envision functionality in new classes of Cyber‐Physical System (CPS) vehicle concepts. Modern day CPS Systems offer opportunities to extract superior performance (reconfigurability, robustness, reliability) from the underlying: (i) articulated‐wheeled electromechanical vehicle platforms (e.g. GroundX vehicles with embodied‐mechatronic preflexes); (ii) multi‐modal multi‐resolution sensor‐based spatio‐temporal information gathering; coupled closely with (iii) size, weight and power (SWaP) constrained algorithmic‐ intelligence to realize “real‐time Sense‐Think‐Act”. This inherent but unexploited capability offers enormous opportunity for superior mobility and information gathering in a range of outdoor terrains.

Intelligent orchestration/coordination of newly provisioned flexibility, from the myriad parameters, settings and behaviors, requires careful evaluation at both design stage and during operations. Our team proposes to develop a Deep Reinforcement Learning Enhanced Decision‐Support Framework for CPSSystem Evaluation enabled by multi‐domain physical and sensor‐intelligence modeling of the ground vehicle. The framework and methods will be applicable across: (i) varied size/weight/architecture CPS vehicles, sensor‐allocation schema, sensor‐intelligence modules, and/or motion planner‐controllers; and (ii) diverse benchmark test‐environments/scenarios. In latter stages of the project the framework will enable simulation‐based design‐refinement and hardware‐in‐the‐loop testing.

Publications:

• Salvi, A., Buzhardt, J., Tallapragada, P., Krovi, V., Brudnak, M., and J. M. Smereka, “Deep reinforcement learning for simultaneous path planning and stabilization of offroad vehicles”, In Proceedings of the Ground Vehicle Systems Engineering and Technology Symposium (GVSETS), NDIA, Novi, MI, Aug. 10-12, 2021.”
• Salvi, A., Buzhardt, J., Tallapragada, P., Krovi, V., Brudnak, M. and J.M. Smereka, “Virtual Evaluation of Deep Learning Techniques for Vision-Based Trajectory Tracking,” SAE Technical Paper 2022-01-0369, 2022, https://doi.org/10.4271/2022-01-0369.
• Joglekar, A., Krovi, V., Brudnak, M., & Smereka, J. M. (2022). Hybrid reinforcement learning based controller for autonomous navigation. Paper presented at the IEEE Vehicular Technology Conference, , 2022-June doi:10.1109/VTC2022-Spring54318.2022.9861014
• Buzhardt, J., & Tallapragada, P. (2022). A koopman operator approach for the vertical stabilization of an off-road vehicle. Paper presented at the IFAC-PapersOnLine, , 55(37) 675-680. doi:10.1016/j.ifacol.2022.11.260
• Salvi, A., Coleman, J., Buzhardt, J., Krovi, V., & Tallapragada, P. (2022). Stabilization of vertical motion of a vehicle on bumpy terrain using deep reinforcement learning∗. Paper presented at the IFAC-PapersOnLine, , 55(37) 276-281. doi:10.1016/j.ifacol.2022.11.197.
• Krolicki, A., Tellez-Castro, D., & Vaidya, U. (2022). Nonlinear dual-mode model predictive control using koopman eigenfunctions. Paper presented at the Proceedings of the IEEE Conference on Decision and Control, , 2022-December 3074-3079. doi:10.1109/CDC51059.2022.9992677
• Ding, F., Yang, Y., Hu, H., Krovi, V., & Luo, F. (2022). Dual-level knowledge distillation via knowledge alignment and correlation. IEEE Transactions on Neural Networks and Learning Systems, , 1-11. doi:10.1109/TNNLS.2022.3190166

#1.A75