Rapid and Adaptive Perception Autonomy for Context-Specific Classifications

Principal Investigators

• Mohammed Haider, University of Alabama at Birmingham (UAB)
• Samuel Misko (co-I), UAB

Faculty

• Steven Gardner, Pierre Fiorini, UAB
• Tulga Ersal, U. of Michigan

Students

• Kamal Hossain, et al, UAB

Government

• Jon Smereka, Paramsothy Jayakumar, Ryan Kreiter, US Army GVSC

Industry

• Arnold Free, Traxara Robotics

Project began Q4 2022.

This project will investigate four research thrusts regarding novel autonomous perception algorithms with the end goal being development of robust perception algorithms that can be used for iterative integration, test, and evaluation as part of the US Army’s Robotic Technology Kernel (RTK), ARC Autonomy Lab, and NATO. The research objectives of the entire project will include

1. To create an adaptive algorithm for synchronized operation of the multi-sensor network in various complex environmental conditions for perception autonomy in unregulated environments.
2. To deploy a customized and efficient reservoir computing scheme as a transfer learning layer with the standard deep-learning platform for utilization of collective information, effective terrain classifications, and cost map generation.
3. To utilize a graph network in a reservoir computing architecture to address both spatial and temporal inference and enable better image segmentation and terrain classification.
4. To create deep Q network by utilizing the custom reservoir computing network for policy extraction with a shorter learning time and limited data set.
5. To utilize temporal inference of the custom reservoir computing network and identification of moving objects rapidly.
6. Power-, memory-, and time-efficient processing algorithms for large-input sensor data.
7. To develop a competitive algorithm from limited representative ground truth datasets.

The four research thrusts (RTs) the proposed project will entail are (i) RT-01: Multi-Sensor Fusion for Adaptive Perception, (ii) RT-02: Graph Reservoir Networks for Spatial-Temporal Correlation, (iii) RT-03: Hybrid Reservoir Computing for Reinforcement Learning, and (iv) RT-04: Rapid Detection of Mobile Agents.

The fundamental research questions the proposed project is going to entail are

1. How to create an algorithm that adapts according to the available sensors and their relative performances in various complex environmental conditions, such that the multi-sensor network can work together in a synchronized manner to perceive unregulated environments.
2. With respect to the machine-learning algorithm, how can transfer learning be employed to improve perception performance? Transfer learning combines multiple algorithms into a single cohesive one, such that it retains memory between frames, can be adapted to accept all sensor types, and utilize collective information to generate output segmentation classifications and cost maps used for local mapping and planning.
3. How a Graph Network-based reservoir can bring its excellent feature in the off-road scenario to address both spatial and temporal inference and enable better image segmentation and terrain classification.
4. Finally, how the inclusion of the ESNs as the Q-network in the DRL framework known as the deep echo state Q-network (DEQN) can bring the excellent benefits of learning a good policy with a short learning time and limited data set [17], which can provide a better and rapid classification of unchartered terrain for autonomous vehicle trafficability.

#1.A100