Dynamic Teaming of Autonomous Vehicles to Address Intelligent Adversarial Actions

Project begins in 2020.

The main goal of this project is to develop AI-based approaches that manage the operation of teams of autonomous vehicles against an intelligent adversary by using specialized attacker-defender games.

We aim to model and design the intelligent behaviors of autonomous vehicle fleets. These intelligent behaviors are enabled by (a) perceiving and processing the battlefield information, (b) inferring the situation and adversarial intentions, and by (c) reacting smartly and collaboratively in teams. Two challenges in military operations which have not been fully addressed by previous studies are considered.

Adaptive learning and communication strategy: Vehicle operation scenarios are diverse and that leads to the different roles for vehicles to perform. These roles are also contextual. The involvement of an intelligent adversary brings in added uncertainty and dynamics to the battlefield environment and creates unexpected and time-variant situations for autonomous vehicle teams. Each vehicle and its team need to keep exploring and learning from imperfect information of the environment and from observed past adversarial activities. Vehicles also need to adaptively adjust their communication strategy for an effective information sharing based on the situation, the role, and the communication constraints.

Misalignment between autonomous vehicle behaviors and human desires: Minimizing such misalignments while maintaining high performance in operations can be accomplished only by AI algorithms that are capable of learning how to behave correctly. Such learning can be designed using adequate reward functions. However, some seemingly reasonable reward functions may create incorrect behaviors. For example, if we reward the action of cleaning up dirt for a vacuum robot, then the converged optimal policy directs the robot to repeatedly dump and clean up the same dirt over and over again. It is not trivial to create an optimal reward function to teach AI algorithms in an attacker-defender game. That is not only because of the delays and unobvious causality in the rewards (e.g., a successful interception of adversarial supplies a week ago may lead to a current success), but also because of adversary-dependent rewards (e.g., performance improvement can be the result of a better own decision or a worse adversarial decision).

In the long term, we plan to use the developed attacker-defender models to evaluate different vehicle designs. That is because autonomous behaviors cannot be thought of in abstraction of vehicle physical attributes. Thus, the design of the vehicle and the design of its operation (including any doctrine) are intimately connected and have to be done simultaneously. The scope of the project is general enough to span a variety of optimization and learning approaches of vehicle action scheduling and communications for teaming considering uncertainty caused by intelligent adversaries.

Publications:

• Haochen Wu, Amin Ghadami, A. Emrah Bayrak, Jonathon M. Smereka, Bogdan I. Epureanu, “Impact of Heterogeneity and Risk Aversion on Task Allocation in Multi-agent Teams”, IEEE Robotics and Automation Letters, Oct 2021.
• Haochen Wu, Amin Ghadami, A. Emrah Bayrak, Jonathon M. Smereka, Bogdan I. Epureanu, “Task Allocation with Load Management in Multi-agent Teams”, IEEE International Conference on Robotics and Automation (ICRA), May 2022.
• Haochen Wu, Charne C. Folks, A. Emrah Bayrak, Jonathon M. Smereka, Bogdan I. Epureanu, “Human- Autonomy Teaming in Immersive Environments”, Interservice/Industry Training, Simulation and Education Conference (I/ITSEC), 2022. (PDF)

Publications from Prior Work Closely Related to the Project:

• Li, X., Arora, K., & Alaniazar, S., 2019. Mixed-Model Text Classification Framework Considering the Practical Constraints (best paper finalist). In 2019 Second International Conference on Artificial Intelligence for Industries (AI4I), IEEE.
• Li, X. & Epureanu, B.I., An Agent-Based Approach for Optimizing Modular Vehicle Fleet Operation. International Journal of Production Economics (under revision).
• Li, X. & Epureanu, B.I., Analysis of Fleet Modularity in an Artificial Intelligence-Based Attacker-Defender Game. European Journal of Operational Research (under revision).
• Li, X., Mitra, M., & Epureanu, B. I., 2019. Analysis of the Synergy between Modularity and Autonomy in an Artificial Intelligence Based Fleet Competition. Proceeding of the 2019 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS).
• Li, X., Nassehi, A., & Epureanu, B. I., 2019. Degradation-aware decision making in reconfigurable manufacturing systems. CIRP Annals-Manufacturing Technology.
• Li, X., Bayrak, A. E., Epureanu, B. I., & Koren, Y., 2018. Real-time teaming of multiple reconfigurable manufacturing systems. CIRP Annals-Manufacturing Technology.
• Bayrak A. E., Egilmez M. M., Kuang H., Li X., Park J. M., Hu J., Papalambros P. Y., Epureanu B. I., Umpfenbach E., Anderson E., & Gorsich D., 2018. A System of Systems Approach to Strategic Feasibility of Modular Vehicle Fleets. IEEE Transactions on Systems, Man and Cybernetics.
• Li, X. & Epureanu, B. I., 2017. Robustness and Adaptability Analysis of Future Military Modular Fleet Operation System. In ASME 2017 Dynamic Systems and Control Conference (pp. V002T05A003).
• Li, X. & Epureanu, B. I., 2017. Intelligent Agent-based Dynamic Scheduling for Military Modular Vehicle Fleets. In IIE Annual Conference Proceedings (pp. 404-409), Institute of Industrial and Systems Engineers.