-
SuperAnimal pretrained pose estimation models for behavioral analysis
Authors:
Shaokai Ye,
Anastasiia Filippova,
Jessy Lauer,
Steffen Schneider,
Maxime Vidal,
Tian Qiu,
Alexander Mathis,
Mackenzie Weygandt Mathis
Abstract:
Quantification of behavior is critical in applications ranging from neuroscience, veterinary medicine and animal conservation efforts. A common key step for behavioral analysis is first extracting relevant keypoints on animals, known as pose estimation. However, reliable inference of poses currently requires domain knowledge and manual labeling effort to build supervised models. We present a serie…
▽ More
Quantification of behavior is critical in applications ranging from neuroscience, veterinary medicine and animal conservation efforts. A common key step for behavioral analysis is first extracting relevant keypoints on animals, known as pose estimation. However, reliable inference of poses currently requires domain knowledge and manual labeling effort to build supervised models. We present a series of technical innovations that enable a new method, collectively called SuperAnimal, to develop unified foundation models that can be used on over 45 species, without additional human labels. Concretely, we introduce a method to unify the keypoint space across differently labeled datasets (via our generalized data converter) and for training these diverse datasets in a manner such that they don't catastrophically forget keypoints given the unbalanced inputs (via our keypoint gradient masking and memory replay approaches). These models show excellent performance across six pose benchmarks. Then, to ensure maximal usability for end-users, we demonstrate how to fine-tune the models on differently labeled data and provide tooling for unsupervised video adaptation to boost performance and decrease jitter across frames. If the models are fine-tuned, we show SuperAnimal models are 10-100$\times$ more data efficient than prior transfer-learning-based approaches. We illustrate the utility of our models in behavioral classification in mice and gait analysis in horses. Collectively, this presents a data-efficient solution for animal pose estimation.
△ Less
Submitted 30 December, 2023; v1 submitted 14 March, 2022;
originally announced March 2022.
-
New views of old proteins: clarifying the enigmatic proteome
Authors:
Participants in a NIH Workshop on Functional,
Integrative Proteomics,
:,
Kristin E. Burnum Johnson,
Thomas P. Conrads,
Richard R. Drake,
Amy E. Herr,
Ravi Iyengar,
Ryan T. Kelly,
Emma Lundberg,
Michael J. MacCoss,
Alexandra Naba,
Garry P. Nolan,
Pavel A. Pevzner,
Karin D. Rodland,
Salvatore Sechi,
Nikolai Slavov,
Jeffrey M. Spraggins,
Jennifer E. Van Eyk,
Marc Vidal,
Christine Vogel,
David R. Walt,
Neil L. Kelleher
Abstract:
All human diseases involve proteins, yet our current tools to characterize and quantify them are limited. To better elucidate proteins across space, time, and molecular composition, we provide provocative projections for technologies to meet the challenges that protein biology presents. With a broad perspective, we discuss grand opportunities to transition the science of proteomics into a more pro…
▽ More
All human diseases involve proteins, yet our current tools to characterize and quantify them are limited. To better elucidate proteins across space, time, and molecular composition, we provide provocative projections for technologies to meet the challenges that protein biology presents. With a broad perspective, we discuss grand opportunities to transition the science of proteomics into a more propulsive enterprise. Extrapolating recent trends, we offer potential futures for a next generation of disruptive approaches to define, quantify and visualize the multiple dimensions of the proteome, thereby transforming our understanding and interactions with human disease in the coming decade.
△ Less
Submitted 17 August, 2021;
originally announced August 2021.
-
Perspectives on individual animal identification from biology and computer vision
Authors:
Maxime Vidal,
Nathan Wolf,
Beth Rosenberg,
Bradley P. Harris,
Alexander Mathis
Abstract:
Identifying individual animals is crucial for many biological investigations. In response to some of the limitations of current identification methods, new automated computer vision approaches have emerged with strong performance. Here, we review current advances of computer vision identification techniques to provide both computer scientists and biologists with an overview of the available tools…
▽ More
Identifying individual animals is crucial for many biological investigations. In response to some of the limitations of current identification methods, new automated computer vision approaches have emerged with strong performance. Here, we review current advances of computer vision identification techniques to provide both computer scientists and biologists with an overview of the available tools and discuss their applications. We conclude by offering recommendations for starting an animal identification project, illustrate current limitations and propose how they might be addressed in the future.
△ Less
Submitted 28 February, 2021;
originally announced March 2021.
