Compression of Unreal Engine Simulations

Behavioral cloning has been widely used to obtain strategies for autonomous driving based on expert demonstrations in virtual environments. However, there are several computational challenges to benefiting from these approaches at scale. Human experts typically consider the full scene context, which includes physical interpretation of depth and objects in the environment. This quantity of data, available from simulated camera and LiDAR data, is challenging to directly provide to a behavioral cloning agent due to its high dimensionality. Second, is the challenge arising from a lack of robustness of the autonomy model to scenarios outside of its training data — necessitating large scale training or the identification of foundational features.

Oakland University has been working on virtual experimentation involving the generation of high-fidelity autonomous vehicle simulation environments and creating highly efficient behavioral cloning models using low-dimensional hand-chosen features only available in simulated environments. Meanwhile, University of Michigan has been working on extracting latent features from large-scale, high-dimensional data using low-rank tensor decomposition algorithms and using those latent features for behavioral cloning in video games.

This integration project aims to merge these two ideas to enable behavioral cloning directly from the large scale high-dimensional sensor data to enable higher performing autonomous agents. We will demonstrate our ability to highly compress the relevant visual and LiDAR data, and our ability to use this data for behavioral cloning. We will then investigate and compare the performance and robustness of agents trained with the low-dimensional data and the original high-dimensional data.