SHREC 2022 Track on Online Detection of Heterogeneous Gestures
Authors:
Ariel Caputo,
Marco Emporio,
Andrea Giachetti,
Marco Cristani,
Guido Borghi,
Andrea D'Eusanio,
Minh-Quan Le,
Hai-Dang Nguyen,
Minh-Triet Tran,
F. Ambellan,
M. Hanik,
E. Nava-Yazdani,
C. von Tycowicz
Abstract:
This paper presents the outcomes of a contest organized to evaluate methods for the online recognition of heterogeneous gestures from sequences of 3D hand poses. The task is the detection of gestures belonging to a dictionary of 16 classes characterized by different pose and motion features. The dataset features continuous sequences of hand tracking data where the gestures are interleaved with non…
▽ More
This paper presents the outcomes of a contest organized to evaluate methods for the online recognition of heterogeneous gestures from sequences of 3D hand poses. The task is the detection of gestures belonging to a dictionary of 16 classes characterized by different pose and motion features. The dataset features continuous sequences of hand tracking data where the gestures are interleaved with non-significant motions. The data have been captured using the Hololens 2 finger tracking system in a realistic use-case of mixed reality interaction. The evaluation is based not only on the detection performances but also on the latency and the false positives, making it possible to understand the feasibility of practical interaction tools based on the algorithms proposed. The outcomes of the contest's evaluation demonstrate the necessity of further research to reduce recognition errors, while the computational cost of the algorithms proposed is sufficiently low.
△ Less
Submitted 22 July, 2022; v1 submitted 14 July, 2022;
originally announced July 2022.
SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild
Authors:
Ariel Caputo,
Andrea Giachetti,
Simone Soso,
Deborah Pintani,
Andrea D'Eusanio,
Stefano Pini,
Guido Borghi,
Alessandro Simoni,
Roberto Vezzani,
Rita Cucchiara,
Andrea Ranieri,
Franca Giannini,
Katia Lupinetti,
Marina Monti,
Mehran Maghoumi,
Joseph J. LaViola Jr,
Minh-Quan Le,
Hai-Dang Nguyen,
Minh-Triet Tran
Abstract:
Gesture recognition is a fundamental tool to enable novel interaction paradigms in a variety of application scenarios like Mixed Reality environments, touchless public kiosks, entertainment systems, and more. Recognition of hand gestures can be nowadays performed directly from the stream of hand skeletons estimated by software provided by low-cost trackers (Ultraleap) and MR headsets (Hololens, Oc…
▽ More
Gesture recognition is a fundamental tool to enable novel interaction paradigms in a variety of application scenarios like Mixed Reality environments, touchless public kiosks, entertainment systems, and more. Recognition of hand gestures can be nowadays performed directly from the stream of hand skeletons estimated by software provided by low-cost trackers (Ultraleap) and MR headsets (Hololens, Oculus Quest) or by video processing software modules (e.g. Google Mediapipe). Despite the recent advancements in gesture and action recognition from skeletons, it is unclear how well the current state-of-the-art techniques can perform in a real-world scenario for the recognition of a wide set of heterogeneous gestures, as many benchmarks do not test online recognition and use limited dictionaries. This motivated the proposal of the SHREC 2021: Track on Skeleton-based Hand Gesture Recognition in the Wild. For this contest, we created a novel dataset with heterogeneous gestures featuring different types and duration. These gestures have to be found inside sequences in an online recognition scenario. This paper presents the result of the contest, showing the performances of the techniques proposed by four research groups on the challenging task compared with a simple baseline method.
△ Less
Submitted 21 June, 2021;
originally announced June 2021.
