Perception in Complex Scenes using Automatically Labeled Sonar-imaging Data in Synthetic Environments

Project began in 2021 and was completed in 2023.

Biological sonar (biosonar) used in the natural world by bats and species of marine mammals shows the tremendous potential of ultrasound for understanding and navigating in complex environments. For example, bats can recognize and hunt millimeter sized flies while navigating in cluttered areas. The ability to create biosonar-like ultrasound beams exists. For example, acoustic phase arrays are used in sonar equipment to generate directive acoustic beams. Moreover, in a previous ARC project, the PIs have created metamaterial-based methods to obtain directive ultrasound beams able to scan the environment in real-time and with very little power usage. The difficulty here lies in interpreting the scattered sound to create maps of the surroundings.

This project aims to augment perception methods for autonomous vehicle navigation by using sonar imaging combined with automatic labeling in synthetic environments for perception. Three fundamental questions will be answered during the execution of this effort:

• How can we simulate sonar-based imaging in synthetic environments such as Unreal Engine (UE4) platforms? (models and tools to simulate sonar in synthetic environments)
• How can sonar-based imaging be used in synthetic environments for labeling of data for training perception algorithms without human supervision (automatic sonar labeling)?
• Can autonomous systems create an accurate world model and classify surrounding objects without compromising stealthiness while operating in any kind of weather?

References:

• B.-I. Popa, B. I. Epureanu, H.-S. Kwon, Long-range ultrasound imaging devices, Provisional Patent, 2020.
• H.-S. Kwon, B. I. Epureanu, and B.-I. Popa, Reconfigurable large-scale bulk metamaterial for broadband ultrasonics, in preparation.

#1.37