Skip to main content
arc logo
Automotive Research Center
hero image
Back to all projects

Vehicle Controls & Behaviors

Annual Plan

Processing Image Data from Unstructured Environments

Project Team

Principal Investigator

Nickolas Vlahopoulos, University of Michigan


Ryan Kreiter, Patel Sahill, Paramsothy Jayakumar, U.S. Army GVSC

Stanton Price, U.S. Army ERDC


William Tecos, General Dynamics

Hongyoon Kim, Samsung Electronics


Spyros Kasapis, University of Michigan

Project Summary

Project began in 2020.

The US Army Ground Vehicle Simulation Center (GVSC) captures a large amount of data from ground vehicle systems (i.e., GPS coordinates, gear changes, user interventions, RPMs, tractive effort, etc.) during development and experimentation in both manned and autonomous operations. Currently, there is a lack of tools for processing unlabeled data in a semantic manner. This missing capability would allow Army engineers to create a clear understanding of the information collected from uncommon and unique events captured during experiments.

The first research objective consists of developing and implementing in PyTorch an Adaptive Deconvolutional Neural Network (ADNN) capability for processing unlabeled images collected from Army battlefield‐like experiments. ADNN has never before been used for softlabeling as it is proposed here; this research will thus produce new fundamental knowledge on how to leverage the ADNN method such that filter weights produced from unlabeled images used for training will benefit the softlabeling performance similar to the ADNN application in unsupervised learning.

The second research objective consists of developing a statistical processor which will analyze the feature maps of the unlabeled images from battlefield experiments and develop clusters of similar statistical significance. The unique aspect of the proposed development is that metadata collected into markdown files from GVSC robotic experiments will be utilized for guiding the clustering for the “critical” type of objects which have been encountered during experiments just before events which have terminated the autonomous operation of a vehicle.

The final research objective will demonstrate how the new capabilities operate with data from GVSC or with other similar data, and transition the new knowledge and code to GVSC. This will be accomplished by having the graduate student supported by this project work at GVSC through summer internships for the duration of the project.


  • Kasapis, S., Zhang, G., Smereka, J., & Vlahopoulos, N. (2020). Using ROC and Unlabeled Data for Increasing Low-Shot Transfer Learning Classification Accuracy. arXiv preprint arXiv:2010.00721.