Vehicle Controls & BehaviorsAnnual Plan
A Robust Semantic-aware Perception System Using Proprioception, Geometry, and Semantics in Unstructured and Unknown Environments
Project begins in 2021.
A real-time robust perception system is a precondition to achieve autonomous off-road mobility at high speed, real-world autonomy, and operation in unstructured and uncertain environments. Today we do not have such a robust perception system capable of supporting complex dynamic off-road missions. Without reliable proprioceptive dead reckoning, the vehicle can be lost and never recovered in perceptually degraded situations, e.g., completely dark, bright, uniform, or foggy scenes. Without a dense, dynamic semantic map, the vehicle’s scene understanding is not sufficient for informed decision-making. Furthermore, autonomous off-road mobility at high speed requires high-frequency and resource-constrained state estimation algorithms that work on-board. The investigators consider proprioception, geometric, and semantics as independent bases that must be considered simultaneously within perception algorithms.
An autonomous off-road vehicle cannot rely on high-definition maps and structured road networks as commercial self-driving vehicles do. Its perception capabilities dictate the behavior of an autonomous vehicle in an unknown environment. This work addresses two core necessities by developing:
- a fail-safe proprioceptive high-frequency state estimator using invariant observer design theory for dead reckoning over long trajectories (i.e., 1 km and above);
- a multilayer semantic mapping framework that models both geometry and semantics of a complex dynamic scene in 3D and runs onboard.
This work addresses the fundamental research questions of:
Q1: What is the performance limit of onboard proprioceptive observers to deal with drifts in the absence of exteroceptive measurements or perceptually degraded situations?
Q2: How to consistently incorporate 3D scene flow estimated from stereo cameras or LIDAR data into a dense semantic map in real-time?