-
Dual-stream spatiotemporal networks with feature sharing for monitoring animals in the home cage
Authors:
Ezechukwu I. Nwokedi,
Rasneer S. Bains,
Luc Bidaut,
Xujiong Ye,
Sara Wells,
James M. Brown
Abstract:
This paper presents a spatiotemporal deep learning approach for mouse behavioural classification in the home-cage. Using a series of dual-stream architectures with assorted modifications to increase performance, we introduce a novel feature sharing approach that jointly processes the streams at regular intervals throughout the network. To investigate the efficacy of this approach, models were eval…
▽ More
This paper presents a spatiotemporal deep learning approach for mouse behavioural classification in the home-cage. Using a series of dual-stream architectures with assorted modifications to increase performance, we introduce a novel feature sharing approach that jointly processes the streams at regular intervals throughout the network. To investigate the efficacy of this approach, models were evaluated by dissociating the streams and training/testing in the same rigorous manner as the main classifiers. Using an annotated, publicly available dataset of a singly-housed mice, we achieve prediction accuracy of 86.47% using an ensemble of a Inception-based network and an attention-based network, both of which utilize this feature sharing. We also demonstrate through ablation studies that for all models, the feature-sharing architectures consistently perform better than conventional ones having separate streams. The best performing models were further evaluated on other activity datasets, both mouse and human. Future work will investigate the effectiveness of feature sharing to behavioural classification in the unsupervised anomaly detection domain.
△ Less
Submitted 3 November, 2022; v1 submitted 1 June, 2022;
originally announced June 2022.
-
Unsupervised detection of mouse behavioural anomalies using two-stream convolutional autoencoders
Authors:
Ezechukwu I Nwokedi,
Rasneer S Bains,
Luc Bidaut,
Sara Wells,
Xujiong Ye,
James M Brown
Abstract:
This paper explores the application of unsupervised learning to detecting anomalies in mouse video data. The two models presented in this paper are a dual-stream, 3D convolutional autoencoder (with residual connections) and a dual-stream, 2D convolutional autoencoder. The publicly available dataset used here contains twelve videos of single home-caged mice alongside frame-level annotations. Under…
▽ More
This paper explores the application of unsupervised learning to detecting anomalies in mouse video data. The two models presented in this paper are a dual-stream, 3D convolutional autoencoder (with residual connections) and a dual-stream, 2D convolutional autoencoder. The publicly available dataset used here contains twelve videos of single home-caged mice alongside frame-level annotations. Under the pretext that the autoencoder only sees normal events, the video data was handcrafted to treat each behaviour as a pseudo-anomaly thereby eliminating them from the others during training. The results are presented for one conspicuous behaviour (hang) and one inconspicuous behaviour (groom). The performance of these models is compared to a single stream autoencoder and a supervised learning model, which are both based on the custom CAE. Both models are also tested on the CUHK Avenue dataset were found to perform as well as some state-of-the-art architectures.
△ Less
Submitted 28 May, 2021;
originally announced June 2021.
-
Integral Curvature Representation and Matching Algorithms for Identification of Dolphins and Whales
Authors:
Hendrik J. Weideman,
Zachary M. Jablons,
Jason Holmberg,
Kiirsten Flynn,
John Calambokidis,
Reny B. Tyson,
Jason B. Allen,
Randall S. Wells,
Krista Hupman,
Kim Urian,
Charles V. Stewart
Abstract:
We address the problem of identifying individual cetaceans from images showing the trailing edge of their fins. Given the trailing edge from an unknown individual, we produce a ranking of known individuals from a database. The nicks and notches along the trailing edge define an individual's unique signature. We define a representation based on integral curvature that is robust to changes in viewpo…
▽ More
We address the problem of identifying individual cetaceans from images showing the trailing edge of their fins. Given the trailing edge from an unknown individual, we produce a ranking of known individuals from a database. The nicks and notches along the trailing edge define an individual's unique signature. We define a representation based on integral curvature that is robust to changes in viewpoint and pose, and captures the pattern of nicks and notches in a local neighborhood at multiple scales. We explore two ranking methods that use this representation. The first uses a dynamic programming time-war** algorithm to align two representations, and interprets the alignment cost as a measure of similarity. This algorithm also exploits learned spatial weights to downweight matches from regions of unstable curvature. The second interprets the representation as a feature descriptor. Feature keypoints are defined at the local extrema of the representation. Descriptors for the set of known individuals are stored in a tree structure, which allows us to perform queries given the descriptors from an unknown trailing edge. We evaluate the top-k accuracy on two real-world datasets to demonstrate the effectiveness of the curvature representation, achieving top-1 accuracy scores of approximately 95% and 80% for bottlenose dolphins and humpback whales, respectively.
△ Less
Submitted 25 August, 2017;
originally announced August 2017.
-
Model-based Catheter Segmentation in MRI-images
Authors:
Andre Mastmeyer,
Guillaume Pernelle,
Lauren Barber,
Steve Pieper,
Dirk Fortmeier,
Sandy Wells,
Heinz Handels,
Tina Kapur
Abstract:
Accurate and reliable segmentation of catheters in MR-guided interventions remains a challenge, and a step of critical importance in clinical workflows. In this work, under reasonable assumptions, mechanical model based heuristics guide the segmentation process allows correct catheter identification rates greater than 98% (error 2.88 mm), and reduction in outliers to one-fourth compared to the sta…
▽ More
Accurate and reliable segmentation of catheters in MR-guided interventions remains a challenge, and a step of critical importance in clinical workflows. In this work, under reasonable assumptions, mechanical model based heuristics guide the segmentation process allows correct catheter identification rates greater than 98% (error 2.88 mm), and reduction in outliers to one-fourth compared to the state of the art. Given distal tips, searching towards the proximal ends of the catheters is guided by mechanical models that are estimated on a per-catheter basis. Their bending characteristics are used to constrain the image feature based candidate points. The final catheter trajectories are hybrid sequences of individual points, each derived from model and image features. We evaluate the method on a database of 10 patient MRI scans including 101 manually segmented catheters. The mean errors were 1.40 mm and the median errors were 1.05 mm. The number of outliers deviating more than 2 mm from the gold standard is 7, and the number of outliers deviating more than 3 mm from the gold standard is just 2.
△ Less
Submitted 10 December, 2020; v1 submitted 18 May, 2017;
originally announced May 2017.
