Mutually-Adaptive Shared Control between Human Operators and Autonomy in Ground Vehicles

Project started in 2018 and is ongoing.

Successful shared control between human operators and autonomy in ground vehicles critically relies on a mutual understanding and adaptation. Various shared control methods have been presented in the literature; however, the metrics that are used to evaluate performance are typically objective engineering metrics and little emphasis has been put so far on the cognitive workload on the human operator. A shared-control paradigm in which the autonomy can adapt to the cognitive workload of the operator could improve mission performance. Such a paradigm, however, is not yet available.

The vision of this collaborative project is that the best negotiation of control and the best long-term system performance will happen only when the human operator and autonomy are mutually aware and adaptive in key aspects involved in control negotiation, including cognitive workload. A system that increases operator workload relative to manual operation may be switched off or underutilized, even if it improves other aspects of performance.

This collaborative project aims to address this gap by developing methods to estimate the cognitive workload of human operators as they interact with autonomy and incorporating those estimations explicitly into the autonomy algorithms. As part of this goal, this project will also investigate the shared control architectures available to date in the literature in terms of their suitability to implement the envisioned mutually adaptive shared control framework, as well as in terms of their suitability to vary and/or quantify the variations in the level of shared control to support GVSC’s needs to evaluate mobility across different levels of shared control.

Publications:

• R. Luo, Y. Wang, Y. Weng, V. Paul, M. J. Brudnak, P. Jayakumar, M. Reed, J. L. Stein, T. Ersal, and J. Yang, “Real-time Assessment of Workload: A Bayesian Inference Approach,” Annual Meeting of the Human Factors and Ergonomics Society, Seattle, Washington, USA, 2019.
• Ruikun Luo, Yifan Weng, Yifan Wang, Paramsothy Jayakumar, Mark J. Brudnak, Victor Paul, Vishnu R. Desaraju, Jeffrey L. Stein, Tulga Ersal, and X. Jessie Yang, “A Workload Adaptive Haptic Shared Control Scheme for Semi-Autonomous Driving”, Accident Analysis and Prevention 152, pp. 105968, 2021.
• Yifan Weng, Ruikun Luo, Paramsothy Jayakumar, Mark Brudnak, Victor Paul, Vishnu Desaraju, Jeffrey L. Stein, Xi Jessie Yang, Tulga Ersal, “Design and Human-in-the-Loop Evaluation of a Workload-Adaptive Haptic Shared Control Framework for Semi-Autonomous Driving”, 2020 American Control Conference, Denver, CO, USA.
• Ruikun Luo, Yifan Weng, Paramsothy Jayakumar, Mark J. Brudnak, Victor Paul, Vishnu R. Desaraju, Jeffrey L. Stein, Tulga Ersal, and X. Jessie Yang, “Real-time Workload Estimation Using Eye Tracking: A Bayesian Inference Approach” International Journal of Human-Computer Interaction (in review)
• Yifan Weng, Ruikun Luo, Paramsothy Jayakumar, Mark J. Brudnak, Victor Paul, Vishnu R. Desaraju, Jeffrey L. Stein, X. Jessie Yang, and Tulga Ersal “A Workload Adaptive Autonomy Formulation for Semi-Autonomous Driving”, Accident Analysis and Prevention (in preparation).
• Yifan Weng, Vishnu R. Desaraju, Jeffrey L. Stein, Tulga Ersal, “Workload Adaptive Haptic Shared Control for Semi-Autonomous Driving: A User Study”, American Control Conference (in preparation).
• R. Luo, Y. Weng, P. Jayakumar, M. J. Brudnak, V. Paul, V. Desaraju, J. L. Stein, T. Ersal, and X. J. Yang, “Real-time workload estimation using eye tracking: A Bayesian inference approach,” International Journal of Human-Computer Interaction, accepted.

Prior publications related to this work:

• M. Y. Zhang and X. J. Yang, “Evaluating effects of workload on automation trust and dependence and task performance,” Proceedings of the 2017 Human Factors and Ergonomics Society Annual Meeting (HFES ’17), Accepted.
• S. Lu, M. Y. Zhang, T. Ersal, and X. J. Yang, “Effects of a delay compensation aid on teleoperation of unmanned ground vehicles” Presented at 2017 UoM Engineering Graduate Symposium, Accepted.
• J. Liu, P. Jayakumar, J. L. Stein, and T. Ersal, “A nonlinear model predictive control formulation for obstacle avoidance in high-speed autonomous ground vehicles in unstructured environments,” Vehicle System Dynamics, accepted.
• J. Liu, P. Jayakumar, J. L. Stein, and T. Ersal, “A Double-Worst-Case Formulation for Improving the Robustness of an MPC-Based Obstacle Avoidance Algorithm to Parametric Uncertainty,” American Control Conference, Seattle, WA, 2017.
• J. Liu, P. Jayakumar, J. L. Stein, and T. Ersal, “Combined Speed and Steering Control in High Speed Autonomous Ground Vehicles for Obstacle Avoidance Using Model Predictive Control,” IEEE Transactions on Vehicular Technology, vol. 66, no. 10, pp. 8746-8763, 2017.
• H. Febbo, J. Liu, P. Jayakumar, J. L. Stein, and T. Ersal, “Moving Obstacle Avoidance for Large, High-Speed Autonomous Ground Vehicles,” American Control Conference, 2017.

#1.27