-
Recurrence over Video Frames (RoVF) for the Re-identification of Meerkats
Authors:
Mitchell Rogers,
Kobe Knowles,
Gaël Gendron,
Shahrokh Heidari,
David Arturo Soriano Valdez,
Mihailo Azhar,
Padriac O'Leary,
Simon Eyre,
Michael Witbrock,
Patrice Delmas
Abstract:
Deep learning approaches for animal re-identification have had a major impact on conservation, significantly reducing the time required for many downstream tasks, such as well-being monitoring. We propose a method called Recurrence over Video Frames (RoVF), which uses a recurrent head based on the Perceiver architecture to iteratively construct an embedding from a video clip. RoVF is trained using…
▽ More
Deep learning approaches for animal re-identification have had a major impact on conservation, significantly reducing the time required for many downstream tasks, such as well-being monitoring. We propose a method called Recurrence over Video Frames (RoVF), which uses a recurrent head based on the Perceiver architecture to iteratively construct an embedding from a video clip. RoVF is trained using triplet loss based on the co-occurrence of individuals in the video frames, where the individual IDs are unavailable. We tested this method and various models based on the DINOv2 transformer architecture on a dataset of meerkats collected at the Wellington Zoo. Our method achieves a top-1 re-identification accuracy of $49\%$, which is higher than that of the best DINOv2 model ($42\%$). We found that the model can match observations of individuals where humans cannot, and our model (RoVF) performs better than the comparisons with minimal fine-tuning. In future work, we plan to improve these models by using pre-text tasks, apply them to animal behaviour classification, and perform a hyperparameter search to optimise the models further.
△ Less
Submitted 18 June, 2024;
originally announced June 2024.
-
RABBIT: A Robot-Assisted Bed Bathing System with Multimodal Perception and Integrated Compliance
Authors:
Rishabh Madan,
Skyler Valdez,
David Kim,
Sujie Fang,
Luoyan Zhong,
Diego Virtue,
Tapomayukh Bhattacharjee
Abstract:
This paper introduces RABBIT, a novel robot-assisted bed bathing system designed to address the growing need for assistive technologies in personal hygiene tasks. It combines multimodal perception and dual (software and hardware) compliance to perform safe and comfortable physical human-robot interaction. Using RGB and thermal imaging to segment dry, soapy, and wet skin regions accurately, RABBIT…
▽ More
This paper introduces RABBIT, a novel robot-assisted bed bathing system designed to address the growing need for assistive technologies in personal hygiene tasks. It combines multimodal perception and dual (software and hardware) compliance to perform safe and comfortable physical human-robot interaction. Using RGB and thermal imaging to segment dry, soapy, and wet skin regions accurately, RABBIT can effectively execute washing, rinsing, and drying tasks in line with expert caregiving practices. Our system includes custom-designed motion primitives inspired by human caregiving techniques, and a novel compliant end-effector called Scrubby, optimized for gentle and effective interactions. We conducted a user study with 12 participants, including one participant with severe mobility limitations, demonstrating the system's effectiveness and perceived comfort. Supplementary material and videos can be found on our website https://emprise.cs.cornell.edu/rabbit.
△ Less
Submitted 26 January, 2024;
originally announced January 2024.
-
Behaviour Modelling of Social Animals via Causal Structure Discovery and Graph Neural Networks
Authors:
Gaël Gendron,
Yang Chen,
Mitchell Rogers,
Yi** Liu,
Mihailo Azhar,
Shahrokh Heidari,
David Arturo Soriano Valdez,
Kobe Knowles,
Padriac O'Leary,
Simon Eyre,
Michael Witbrock,
Gillian Dobbie,
Jiamou Liu,
Patrice Delmas
Abstract:
Better understanding the natural world is a crucial task with a wide range of applications. In environments with close proximity between humans and animals, such as zoos, it is essential to better understand the causes behind animal behaviour and what interventions are responsible for changes in their behaviours. This can help to predict unusual behaviours, mitigate detrimental effects and increas…
▽ More
Better understanding the natural world is a crucial task with a wide range of applications. In environments with close proximity between humans and animals, such as zoos, it is essential to better understand the causes behind animal behaviour and what interventions are responsible for changes in their behaviours. This can help to predict unusual behaviours, mitigate detrimental effects and increase the well-being of animals. There has been work on modelling the dynamics behind swarms of birds and insects but the complex social behaviours of mammalian groups remain less explored. In this work, we propose a method to build behavioural models using causal structure discovery and graph neural networks for time series. We apply this method to a mob of meerkats in a zoo environment and study its ability to predict future actions and model the behaviour distribution at an individual-level and at a group level. We show that our method can match and outperform standard deep learning architectures and generate more realistic data, while using fewer parameters and providing increased interpretability.
△ Less
Submitted 21 December, 2023;
originally announced December 2023.
-
Meerkat Behaviour Recognition Dataset
Authors:
Mitchell Rogers,
Gaël Gendron,
David Arturo Soriano Valdez,
Mihailo Azhar,
Yang Chen,
Shahrokh Heidari,
Caleb Perelini,
Padriac O'Leary,
Kobe Knowles,
Izak Tait,
Simon Eyre,
Michael Witbrock,
Patrice Delmas
Abstract:
Recording animal behaviour is an important step in evaluating the well-being of animals and further understanding the natural world. Current methods for documenting animal behaviour within a zoo setting, such as scan sampling, require excessive human effort, are unfit for around-the-clock monitoring, and may produce human-biased results. Several animal datasets already exist that focus predominant…
▽ More
Recording animal behaviour is an important step in evaluating the well-being of animals and further understanding the natural world. Current methods for documenting animal behaviour within a zoo setting, such as scan sampling, require excessive human effort, are unfit for around-the-clock monitoring, and may produce human-biased results. Several animal datasets already exist that focus predominantly on wildlife interactions, with some extending to action or behaviour recognition. However, there is limited data in a zoo setting or data focusing on the group behaviours of social animals. We introduce a large meerkat (Suricata Suricatta) behaviour recognition video dataset with diverse annotated behaviours, including group social interactions, tracking of individuals within the camera view, skewed class distribution, and varying illumination conditions. This dataset includes videos from two positions within the meerkat enclosure at the Wellington Zoo (Wellington, New Zealand), with 848,400 annotated frames across 20 videos and 15 unannotated videos.
△ Less
Submitted 20 June, 2023;
originally announced June 2023.
-
Information Seeking and Information Processing Behaviors Among Type 2 Diabetics
Authors:
Sarah Masud Preum,
Kate Clark,
Ashley Davis,
Konstantine Khutsishvilli,
Rupa S Valdez
Abstract:
Effective patient education is critical for managing Type 2 Diabetes Mellitus (T2DM), one of the most common chronic diseases in the United States. While some studies focus on the information-seeking behavior of T2DM patients, other self-education behaviors including information processing and utilization are rarely explored in the context of T2DM. This study sought to assess two self-education be…
▽ More
Effective patient education is critical for managing Type 2 Diabetes Mellitus (T2DM), one of the most common chronic diseases in the United States. While some studies focus on the information-seeking behavior of T2DM patients, other self-education behaviors including information processing and utilization are rarely explored in the context of T2DM. This study sought to assess two self-education behaviors of type 2 diabetics, namely, information seeking and information processing, to understand more about how these behaviors affect the self-management of this common chronic disease. Semi-structured interviews were conducted with 8 English speaking T2DM patients and qualitative content analysis techniques were performed to analyze their responses. The information seeking and processing behaviors vary across individuals based on their prognosis of T2DM, information needs, and personal preferences. Patients are often dissatisfied with information from official sources, have difficulty evaluating the trustworthiness of information sources, and desire information that is more personally relevant to them. Several participants identified a lack of personalized information as a key factor in the inability to adhere to T2DM management guidelines, which led them to experience increased glucose levels, difficulty managing A1C levels, frustration, and anxiety. They mentioned that they followed trial and error based approaches to tailor information according to their needs and physiological conditions. Many participants identified conflicting or inconsistent information from different sources as a major barrier to information processing. The results of this study indicate a need for authentic, consistent, and individualized information for type 2 diabetics.
△ Less
Submitted 28 October, 2019;
originally announced October 2019.
-
An Evolutionary Algorithm of Linear complexity: Application to Training of Deep Neural Networks
Authors:
S. Ivvan Valdez,
Alfonso Rojas-Domínguez
Abstract:
The performance of deep neural networks, such as Deep Belief Networks formed by Restricted Boltzmann Machines (RBMs), strongly depends on their training, which is the process of adjusting their parameters. This process can be posed as an optimization problem over n dimensions. However, typical networks contain tens of thousands of parameters, making this a High-Dimensional Problem (HDP). Although…
▽ More
The performance of deep neural networks, such as Deep Belief Networks formed by Restricted Boltzmann Machines (RBMs), strongly depends on their training, which is the process of adjusting their parameters. This process can be posed as an optimization problem over n dimensions. However, typical networks contain tens of thousands of parameters, making this a High-Dimensional Problem (HDP). Although different optimization methods have been employed for this goal, the use of most of the Evolutionary Algorithms (EAs) becomes prohibitive due to their inability to deal with HDPs. For instance, the Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES) which is regarded as one of the most effective EAs, exhibits the enormous disadvantage of requiring $O(n^2)$ memory and operations, making it unpractical for problems with more than a few hundred variables. In this paper, we introduce a novel EA that requires $O(n)$ operations and memory, but delivers competitive solutions for the training stage of RBMs with over one million variables, when compared against CMA-ES and the Contrastive Divergence algorithm, which is the standard method for training RBMs.
△ Less
Submitted 12 July, 2019;
originally announced July 2019.
