Automotive Research Center
Automotive Research Center
A U.S. Army Center of Excellence for Modeling & Simulation of Ground Vehicles
Led by the University of Michigan

Teleoperation with Semi-Autonomous Behaviors and Latency

This project began in 2014 and was completed in 2016.

Schematic of closed loop teleoperated robotic systemThe goal of this project is to improve the overall performance of the closed loop teleoperated robotic system as shown in the figure to the left. Based on current limitations of directly-controlled (e.g. human operator inputs of steering and throttle), teleoperated robots, we know that autonomous behaviors must be included. However, it is important to understand how the user interacts with these autonomous (or semi-autonomous) behaviors. Direct control of teleoperated robots extends the warfighter’s mobility into areas that are too difficult or dangerous to reach. The addition of autonomous behaviors adds intelligence to this mobility, increasing the capabilities of the human-robot team.

Building on a previous ARC project, where a mathematical model that can characterize user behavior in a direct-control teleoperation steering task with latency, we expand our understanding of latency’s effect on user performance in teleoperation tasks, including under semi-autonomous behaviors. A good understanding of the user behavior remains important for characterizing the closed-loop performance.

The research objective of this project is to model and characterize the closed-loop performance of a user teleoperating a mobile robot with significant latency, with the objective of developing design guidelines that can maximize the overall performance. Towards this end, we developed and implemented semi-autonomous behaviors on a simulated mobile robot (using the ANVEL software), developed different control blending methods to combine the user’s commands with the autonomy, and performed user tests to assess the performance. The major research questions addressed include:

  1. How should the user and autonomous control inputs be blended to give the best overall "shared control" performance of the teleoperated mobile robot?
  2. How do factors unique to teleoperation (such as latency and highly unstructured environments) affect the optimal blending method?
  3. How should the system performance be defined for teleoperation in an unstructured environment? What is the objective that should be optimized?

This project leveraged related research in the ARC.



  1. Storms, S. Vozar, and D. Tilbury, "Predicting Human Performance During Teleoperation," ACM/IEEE International Conference on Human-Robot Interaction, pp. 298-299, 2014.
  2. J. Storms and D. Tilbury, "Blending of Human and Obstacle Avoidance Control for a High Speed Mobile Robot," American Control Conference, 2014.
  3. J. Storms, and D. Tilbury, "Reducing Rollover Risk of a High Speed Mobile Manipulator," in ASME Dynamic Systems and Controls Conference, 2014.
  4. J. Storms, and D. Tilbury, "Equating User Performance Among Communication Latency Distributions and Simulation Fidelities," IEEE International Conference on Robotics and Automation, 2015.
  5. J. Storms, and D. Tilbury. "Dynamic Weight-Shifting for Improved Rollover Prevention and Handling Dynamics in High Speed Mobile Manipulators," ASME Journal of Dynamic Systems, Measurement, and Control, 2016.
  6. J. Storms, K. Chen, and D. Tilbury, "A Semi-Autonomous Control Method to Improve Performance of Small Unmanned Ground Vehicles with Communication Latency," ASME International Design Engineering Technical Conferences, 2016.
  7. S. Vozar, J. Storms, and D. Tilbury, "Development and Analysis of an Operator Steering Model for Teleoperated Mobile Robots Under Constant and Variable Latencies," Robotica, 2016.
  8. J. Storms, K. Chen, and D. Tilbury, "A Shared Control Method for Obstacle Avoidance with Mobile Robots and its Interaction with Communication Delay," International Journal of Robotics Research, 2016.
  9. J. Storms, K. Chen, D. Tilbury, "Modeling Teleoperated Robot Performance as a Function of Environment Difficulty," IFAC Conference on Cyber Physical and Human Systems, 2016.
  10. J. Storms, D. Tilbury, "A New Difficulty Index for Teleoperated Robots Driving through Obstacles: How Environment Difficulty, Communication Delay and Autonomy Level Affect Performance," Journal of Intelligent and Robotic Systems, Accepted June 2, 2017.
  11. J. Storms, "Modeling and Improving Teleoperation Performance of Semi-Autonomous Wheeled Robots," PhD Dissertation, University of Michigan, 2016.



Visit us | Contact us

Sponsored by U.S. Army CCDC Ground Vehicle Systems Center
Cooperative Agreement W56HZV-19-2-0001

© All rights reserved
Automotive Research Center