Mutually-Adaptive Shared Control between Human Operators and Autonomy in Ground Vehicles

Project started in 2018 and was completed Q2 2021.

Successful shared control between human operators and autonomy in ground vehicles critically relies on a mutual understanding and adaptation. Various shared control methods have been presented in the literature; however, the metrics that are used to evaluate performance are typically objective engineering metrics and little emphasis has been put so far on the cognitive workload on the human operator. A shared-control paradigm in which the autonomy can adapt to the cognitive workload of the operator could improve mission performance. Such a paradigm, however, is not yet available.

The vision of this collaborative project is that the best negotiation of control and the best long-term system performance will happen only when the human operator and autonomy are mutually aware and adaptive in key aspects involved in control negotiation, including cognitive workload. A system that increases operator workload relative to manual operation may be switched off or underutilized, even if it improves other aspects of performance.

This collaborative project aims to address this gap by developing methods to estimate the cognitive workload of human operators as they interact with autonomy and incorporating those estimations explicitly into the autonomy algorithms. As part of this goal, this project will also investigate the shared control architectures available to date in the literature in terms of their suitability to implement the envisioned mutually adaptive shared control framework, as well as in terms of their suitability to vary and/or quantify the variations in the level of shared control to support GVSC’s needs to evaluate mobility across different levels of shared control.

Publications:

• R. Luo, Y. Wang, Y. Weng, V. Paul, M. J. Brudnak, P. Jayakumar, M. Reed, J. L. Stein, T. Ersal, and J. Yang, “Real-time Assessment of Workload: A Bayesian Inference Approach,” Annual Meeting of the Human Factors and Ergonomics Society, Seattle, Washington, USA, 2019.
• Yifan Weng, Ruikun Luo, Paramsothy Jayakumar, Mark Brudnak, Victor Paul, Vishnu Desaraju, Jeffrey L. Stein, Xi Jessie Yang, Tulga Ersal, “Design and Human-in-the-Loop Evaluation of a Workload-Adaptive Haptic Shared Control Framework for Semi-Autonomous Driving”, 2020 American Control Conference, Denver, CO, USA.
• Luo, Ruikun, Yifan Weng, Yifan Wang, Paramsothy Jayakumar, Mark J. Brudnak, Victor Paul, Vishnu R. Desaraju, Jeffrey L. Stein, Tulga Ersal, and X. Jessie Yang. “A workload adaptive haptic shared control scheme for semi-autonomous driving.” Accident Analysis & Prevention 152 (2021): 105968.
• Luo, Ruikun, Yifan Weng, Paramsothy Jayakumar, Mark J. Brudnak, Victor Paul, Vishnu R. Desaraju, Jeffrey L. Stein, Tulga Ersal, and X. Jessie Yang. “Real-time workload estimation using eye tracking: A Bayesian inference approach.” International Journal of Human–Computer Interaction 40, no. 15 (2024): 4042-4057.
• Luo, R., Weng, Y., Jayakumar, P., Brudnak, M. J., Paul, V., Desaraju, V. R., … & Yang, X. J. (2024). Real-time workload estimation using eye tracking: A Bayesian inference approach. International Journal of Human–Computer Interaction, 40(15), 4042-4057.

#1.27