A major challenge for autonomous vehicles is when an AV encounters a flagman who regulates traffic near a construction workzone area or at an automotive crash site. In such real-world scenarios, recognizing the flagman's gesture is an essential AV function that has not received much attention. A key piece of earlier work used a chroma-key (green background) screen to generate datasets and evaluate the accuracy of flagman gesture recognition. We find that this background-agnostic approach drops significantly in performance when the chroma-key background is replaced with images from real-world traffic scenarios. In this paper, we extend that baseline approach by adding contextual information and boosting the baseline's accuracy and robustness. First, we replace the chroma-key background to create virtually augmented (VA) images to capture real-world contexts. Next, we propose and evaluate three approaches: (1) Extract CNN features directly from the “raw” VA images, (2) Extracting CNN features after embedding flagman skeleton and prop information in the VA images, and (3) Utilize an attention mechanism via a transformer with padding masks. All three approaches outperform the baseline method. Notably, our Two-Stage Classifier with Transformer approach boosts the baseline's performance from an F-score of 32% to 80% in challenging VA traffic scenarios.
Enhancing Flagman Gesture Recognition with Virtual Augmentation
08.10.2022
2309706 byte
Aufsatz (Konferenz)
Elektronische Ressource
Englisch
Electronic Flagman Vehicle Approach Warning System
British Library Conference Proceedings | 1991
|Control Virtual Human with Speech Recognition and Gesture Recognition Technology
Springer Verlag | 2012
|GESTURE RECOGNITION DEVICE AND GESTURE RECOGNITION METHOD
Europäisches Patentamt | 2022
|GESTURE RECOGNITION ASSISTANCE APPARATUS AND GESTURE RECOGNITION METHOD
Europäisches Patentamt | 2024
|