Computer Vision for Primate Behavior Analysis in the Wild
Authors:
Richard Vogg,
Timo Lüddecke,
Jonathan Henrich,
Sharmita Dey,
Matthias Nuske,
Valentin Hassler,
Derek Murphy,
Julia Fischer,
Julia Ostner,
Oliver Schülke,
Peter M. Kappeler,
Claudia Fichtel,
Alexander Gail,
Stefan Treue,
Hansjörg Scherberger,
Florentin Wörgötter,
Alexander S. Ecker
Abstract:
Advances in computer vision as well as increasingly widespread video-based behavioral monitoring have great potential for transforming how we study animal cognition and behavior. However, there is still a fairly large gap between the exciting prospects and what can actually be achieved in practice today, especially in videos from the wild. With this perspective paper, we want to contribute towards…
▽ More
Advances in computer vision as well as increasingly widespread video-based behavioral monitoring have great potential for transforming how we study animal cognition and behavior. However, there is still a fairly large gap between the exciting prospects and what can actually be achieved in practice today, especially in videos from the wild. With this perspective paper, we want to contribute towards closing this gap, by guiding behavioral scientists in what can be expected from current methods and steering computer vision researchers towards problems that are relevant to advance research in animal behavior. We start with a survey of the state-of-the-art methods for computer vision problems that are directly relevant to the video-based study of animal behavior, including object detection, multi-individual tracking, (inter)action recognition and individual identification. We then review methods for effort-efficient learning, which is one of the biggest challenges from a practical perspective. Finally, we close with an outlook into the future of the emerging field of computer vision for animal behavior, where we argue that the field should move fast beyond the common frame-by-frame processing and treat video as a first-class citizen.
△ Less
Submitted 29 January, 2024;
originally announced January 2024.
Deep Learning for real-time neural decoding of grasp
Authors:
Paolo Viviani,
Ilaria Gesmundo,
Elios Ghinato,
Andres Agudelo-Toro,
Chiara Vercellino,
Giacomo Vitali,
Letizia Bergamasco,
Alberto Scionti,
Marco Ghislieri,
Valentina Agostini,
Olivier Terzo,
Hansjörg Scherberger
Abstract:
Neural decoding involves correlating signals acquired from the brain to variables in the physical world like limb movement or robot control in Brain Machine Interfaces. In this context, this work starts from a specific pre-existing dataset of neural recordings from monkey motor cortex and presents a Deep Learning-based approach to the decoding of neural signals for grasp type classification. Speci…
▽ More
Neural decoding involves correlating signals acquired from the brain to variables in the physical world like limb movement or robot control in Brain Machine Interfaces. In this context, this work starts from a specific pre-existing dataset of neural recordings from monkey motor cortex and presents a Deep Learning-based approach to the decoding of neural signals for grasp type classification. Specifically, we propose here an approach that exploits LSTM networks to classify time series containing neural data (i.e., spike trains) into classes representing the object being grasped. The main goal of the presented approach is to improve over state-of-the-art decoding accuracy without relying on any prior neuroscience knowledge, and leveraging only the capability of deep learning models to extract correlations from data. The paper presents the results achieved for the considered dataset and compares them with previous works on the same dataset, showing a significant improvement in classification accuracy, even if considering simulated real-time decoding.
△ Less
Submitted 2 November, 2023;
originally announced November 2023.
