Processing Image Data from Unstructured Environments

Project Team

Principal Investigator

Nickolas Vlahopoulos, University of Michigan

Government

Ryan Kreiter, Patel Sahill, Paramsothy Jayakumar, U.S. Army GVSC

Stanton Price, U.S. Army ERDC

Industry

William Tecos, General Dynamics

Hongyoon Kim, Samsung Electronics

Student

Spyros Kasapis, University of Michigan

Project Summary

Project began in 2020 and reached completion Q1 2023.

The US Army Ground Vehicle Simulation Center (GVSC) captures a large amount of data from ground vehicle systems (i.e., GPS coordinates, gear changes, user interventions, RPMs, tractive effort, etc.) during development and experimentation in both manned and autonomous operations. Currently, there is a lack of tools for processing unlabeled data in a semantic manner. This missing capability would allow Army engineers to create a clear understanding of the information collected from uncommon and unique events captured during experiments.

The first research objective consisted of developing and implementing in PyTorch an Adaptive Deconvolutional Neural Network (ADNN) capability for processing unlabeled images collected from Army battlefield‐like experiments. ADNN has never before been used for softlabeling as it is proposed here; this research thus produced new fundamental knowledge on how to leverage the ADNN method such that filter weights produced from unlabeled images used for training will benefit the softlabeling performance similar to the ADNN application in unsupervised learning.

The second research objective consisted of developing a statistical processor which analyzes the feature maps of the unlabeled images from battlefield experiments and develop clusters of similar statistical significance. The unique aspect of the development is that metadata collected into markdown files from GVSC robotic experiments is utilized for guiding the clustering for the “critical” type of objects which have been encountered during experiments just before events which have terminated the autonomous operation of a vehicle.

The final research objective demonstrated how the new capabilities operate with data from GVSC or with other similar data, and transition the new knowledge and code to GVSC. This was accomplished by having the graduate student supported by this project work at GVSC through summer internships for the duration of the project.

Publications:

Kasapis, S., Zhang, G., Smereka, J., & Vlahopoulos, N. (2020). Using ROC and Unlabeled Data for Increasing Low-Shot Transfer Learning Classification Accuracy. arXiv preprint arXiv:2010.00721.
Spiridon Kasapis, Geng Zang, Jonathon M. Smereka, and Nickolas Vlahopoulos, “Using Unlabeled Data for Increasing Low-Shot Classification Accuracy of Relevant and Open-Set Irrelevant Images”, GJCST, vol. 22, no. D2, pp. 11–24, May 2022.
Kasapis S, Zhang G, Smereka JM, Vlahopoulos N. Open-set low-shot classification leveraging the power of instance discrimination. The Journal of Defense Modeling and Simulation. 2022;0(0).