2023 ARC Research Seminar - Fall Series

PI: Dr. Christopher Goodin (Mississippi State U.)
project link

Abstract: How much force is required for an autonomous ground vehicle (AGV) to override clumps of shrubs, thick grasses, or even small trees? This question is critically important for effective AGV navigation in off-road environments. However, while override has been measured extensively for large military vehicles overriding trees (>1 inch diameter), there is almost no information on the forces exerted on intermediate sized (MRZR, Warthog, etc) vehicles overriding clumps of small vegetation. To tackle this problem, we created the first dataset measuring these intermediate override forces by using novel integrated “touch-based” sensing. Next, we are using machine learning to develop a predictive model of the magnitude of the force using remotely measured sensor data like images and lidar point clouds. In this talk I will show the initial results from the first year of the project. We conducted over 80 override tests on our off-road proving ground and developed an initial predictive model by retraining GoogleNet with the data we acquired. Our initial results are promising for images, and we will use sensor fusion in year 2 to improve predictions.

PIs: Dr. Maani Ghaffari, Dr. Kira Barton (U. of Michigan)
project link

Abstract: In this talk, I'll discuss the state-of-the-art findings on off-road real-time semantic mapping. We develop a modular neural network for real-time semantic mapping in uncertain environments, which explicitly updates per-voxel probabilistic distributions within a neural network layer. Our approach combines the reliability of classical probabilistic algorithms with the performance and efficiency of modern neural networks. Although robotic perception is often divided between modern differentiable methods and classical explicit methods, a union of both is necessary for real-time and trustworthy performance. We introduce a novel Convolutional Bayesian Kernel Inference (ConvBKI) layer, which incorporates semantic segmentation predictions online into a 3D map through a depthwise convolution layer by leveraging conjugate priors. We compare ConvBKI against state-of-the-art deep learning approaches and probabilistic algorithms for mapping to evaluate reliability and performance. We also create a Robot Operating System (ROS) package of ConvBKI and test it on real-world perceptually challenging off-road driving data.

Shay Snyder, Maryam Parsa (George Mason U.)
project link

Abstract: How can we achieve collective intelligence by integrating shared experiences among diverse individuals in multi-agent environments while distributing computation among these agents? This question holds significant importance, especially in the context of battlefield-ready robotic systems, where minimizing centralized knowledge is crucial. In this presentation, we introduce an innovative framework designed to achieve Distributed Collective Intelligence (DCI) using multi-level vector symbolic architectures (VSAs). This comprehensive framework encodes an extensive array of sensory data, ranging from inputs derived from RGB cameras, infrared sensors, temperature sensors, energy meters, and more. Furthermore, it seamlessly accommodates arbitrary time-series data and probabilistic knowledge by representing them as symbolic vectors. During this talk, we delve into the mathematical foundations of this hyperdimensional architecture and provide visual demonstrations of its properties. We will also explore the broad applicability of this framework, showcasing its potential as a probabilistic programming language. Additionally, we present preliminary results that highlight the probabilistic mapping capabilities of this system, comparing them to existing methods in single-agent occupancy grid mapping. We also outline our plans for expanding this work to distributed multi-agent mapping.

Jungyun Lim, Lorenzo Valdevit, Diran Apelian (U. of California, Irvine)
project link

Abstract: Structures in military ground vehicles must protect personnel and critical electronic components from impact and severe vibrations. While the absence of personnel in autonomous vehicles presents an opportunity for more aggressive mission profiles (e.g., faster driving on challenging terrain), the corresponding increase in vibration and impact loads poses challenges to the delicate electronic components. As in unmanned vehicles electronic failure almost always implies loss of the asset, the development of next-generation structures and materials for vibration isolation and impact protection becomes paramount. Traditional designs for packaging of electronics use metallic structures for stiffness and strength, elastomeric connections to mitigate peak accelerations resulting from vibrations and impacts, and actively cooled heat sinks to control the chip temperature. While adequate for common situations, these designs suffer from multiple limitations in aggressive scenarios. Here we explore a fully integrated all-metallic lightweight solution to provide mechanical integrity, vibration isolation, impact protection and active cooling of electronic components for autonomous ground vehicles. The proposed system will consist of a highly engineered additively manufactured aluminum metamaterial. As a preliminary step towards this vision, we present a fully-metallic resonant metamaterial concept which offers tunable vibration isolation at low frequencies and impact protection.

PI: Dr. Sang-Hwan Kim (U. of Michigan, Dearborn)
project link

Abstract: In Human-AI Teaming (HAT), the enhancement of situational awareness and trust plays a pivotal role in elevating task performance. Achieving this enhancement involves the effective provision of transparency, elucidating the actions and rationale of the AI system. This study is to explore the features of HAT displays that offer optimal transparency information, specifically in dynamic, multitasking situations such as battle, where AI supports human operators. This presentation introduces the outcomes of the first-year study, including the development of a dynamic dual-task simulation platform and the results of the initial experiment aiming to discern natural communication methods among human operators collaborating on simulation tasks. Ten Pairs of participants assumed the roles of main operators and virtual AI assistants, engaging in the experiment. Types of collaboration strategies, behaviors, and communication forms based on natural interactions were identified through the experiment. The observed behavioral patterns and communication methods will be integrated into the AI and communication formats in the second-year study, contributing to the determination of optimal transparency features in the task scenarios.

PIs: Drs. Ali Beheshti and Shaghayegh (Shay) Bagheri, (George Mason U.)
project link

Abstract: Ground vehicle fuel tanks are equipped with self-sealing bladders to limit fuel loss when a small ballistic threat pierces through them. The self-sealing bladder liner consists of a multilayer coating system containing a polymer matrix and embedded beads. Upon contact with fuel in the tank, the polymeric beads swell significantly, facilitating hole closure and minimizing fuel leakage. Literature lacks basic material properties data on the core components of the coating system as well as fundamental approaches to model the phenomenon. As such, phenomenological approaches based on fundamental material characteristics and through experiments along with simulations are required to enable fuel tank M&S methodology. Accordingly, this project aims to extract the material properties of the polymer system and plans to use a mechanistic-based finite element simulation to model the phenomenon. The study will enhance the predictive modeling capabilities of the fuel tank liners leading to future design enhancements. It will also enable fuel loss estimation, and hence, prediction of vehicles' operational longevity following a ballistic event. During the upcoming presentation, we will cover the first-year results focusing on understanding the sealing mechanism as well as fundamental material characteristics through experimental tests.

PIs: Dr. Tulga Ersal, Dr. Bogdan Epureanu (U. of Michigan)
project link

Abstract: Autonomous vehicles are currently unable to operate on steep off-road slopes, limiting their performance in military applications. The steepness of the terrain necessitates dynamic maneuvers at high speed, yet the roughness of the terrain makes such operation dangerous, altogether limiting the space of feasible trajectories. Additionally, off-road terrains are typically composed of loose soils, whose properties are known only with uncertainty, if at all, exacerbating the challenge. This talk presents our vision for a novel local trajectory planning architecture, capable of controlling an autonomous off-road vehicle on steep, rugged terrain despite uncertainty. We’ll lay out the new combination of model predictive control, Monte Carlo simulation, and learning strategies that form our proposed architecture and highlight some of the progress made to date in understanding the role of model fidelity in the context of open-loop accuracy, closed-loop performance, and computational cost.

PI: Dr. Mohammad Al Faruque; Presenter: Trier Mortlock (U. of California, Irvine)
project link

Abstract: How can adaptive sensor fusion algorithms enhance the perception performance of an autonomous ground vehicle (AGV) deployed with multiple heterogeneous sensors? During AGV operations in challenging, stochastic environments, some sensing modalities negatively impact perception while increasing energy consumption and latency which can be detrimental to safety-critical missions. Existing perception methods are insufficiently robust in harsh operating environments (e.g., extreme terrain, high speeds, bad weather, low light, sensor obstructions) due to (i) rigidity in their fusion implementations, (ii) their inability to model contextual information in real-time, and (iii) the high levels of sensor input uncertainties present. In this talk, we describe our latest efforts to design adaptive sensor fusion methods that can overcome these challenges by leveraging state-of-the-art computer vision models and ensemble learning techniques. We present new theories on the formation of deep learning ensembles and show results across four datasets featuring challenging sensing conditions and two AV perception tasks, object detection and semantic segmentation.