-
Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model's Generalizability in Permafrost Map**
Authors:
Wenwen Li,
Chia-Yu Hsu,
Sizhe Wang,
Yezhou Yang,
Hyunho Lee,
Anna Liljedahl,
Chandi Witharana,
Yili Yang,
Brendan M. Rogers,
Samantha T. Arundel,
Matthew B. Jones,
Kenton McHenry,
Patricia Solis
Abstract:
This paper assesses trending AI foundation models, especially emerging computer vision foundation models and their performance in natural landscape feature segmentation. While the term foundation model has quickly garnered interest from the geospatial domain, its definition remains vague. Hence, this paper will first introduce AI foundation models and their defining characteristics. Built upon the…
▽ More
This paper assesses trending AI foundation models, especially emerging computer vision foundation models and their performance in natural landscape feature segmentation. While the term foundation model has quickly garnered interest from the geospatial domain, its definition remains vague. Hence, this paper will first introduce AI foundation models and their defining characteristics. Built upon the tremendous success achieved by Large Language Models (LLMs) as the foundation models for language tasks, this paper discusses the challenges of building foundation models for geospatial artificial intelligence (GeoAI) vision tasks. To evaluate the performance of large AI vision models, especially Meta's Segment Anything Model (SAM), we implemented different instance segmentation pipelines that minimize the changes to SAM to leverage its power as a foundation model. A series of prompt strategies was developed to test SAM's performance regarding its theoretical upper bound of predictive accuracy, zero-shot performance, and domain adaptability through fine-tuning. The analysis used two permafrost feature datasets, ice-wedge polygons and retrogressive thaw slumps because (1) these landform features are more challenging to segment than manmade features due to their complicated formation mechanisms, diverse forms, and vague boundaries; (2) their presence and changes are important indicators for Arctic warming and climate change. The results show that although promising, SAM still has room for improvement to support AI-augmented terrain map**. The spatial and domain generalizability of this finding is further validated using a more general dataset EuroCrop for agricultural field map**. Finally, we discuss future research directions that strengthen SAM's applicability in challenging geospatial domains.
△ Less
Submitted 16 January, 2024;
originally announced January 2024.
-
The weird and the wonderful in our Solar System: Searching for serendipity in the Legacy Survey of Space and Time
Authors:
Brian Rogers,
Chris J. Lintott,
Steve Croft,
Megan E. Schwamb,
James R. A. Davenport
Abstract:
We present a novel method for anomaly detection in Solar System object data, in preparation for the Legacy Survey of Space and Time. We train a deep autoencoder for anomaly detection and use the learned latent space to search for other interesting objects. We demonstrate the efficacy of the autoencoder approach by finding interesting examples, such as interstellar objects, and show that using the…
▽ More
We present a novel method for anomaly detection in Solar System object data, in preparation for the Legacy Survey of Space and Time. We train a deep autoencoder for anomaly detection and use the learned latent space to search for other interesting objects. We demonstrate the efficacy of the autoencoder approach by finding interesting examples, such as interstellar objects, and show that using the autoencoder, further examples of interesting classes can be found. We also investigate the limits of classic unsupervised approaches to anomaly detection through the generation of synthetic anomalies and evaluate the feasibility of using a supervised learning approach. Future work should consider expanding the feature space to increase the variety of anomalies that can be uncovered during the survey using an autoencoder.
△ Less
Submitted 16 January, 2024;
originally announced January 2024.
-
A Step Towards Efficient Evaluation of Complex Perception Tasks in Simulation
Authors:
Jonathan Sadeghi,
Blaine Rogers,
James Gunn,
Thomas Saunders,
Sina Samangooei,
Puneet Kumar Dokania,
John Redford
Abstract:
There has been increasing interest in characterising the error behaviour of systems which contain deep learning models before deploying them into any safety-critical scenario. However, characterising such behaviour usually requires large-scale testing of the model that can be extremely computationally expensive for complex real-world tasks. For example, tasks involving compute intensive object det…
▽ More
There has been increasing interest in characterising the error behaviour of systems which contain deep learning models before deploying them into any safety-critical scenario. However, characterising such behaviour usually requires large-scale testing of the model that can be extremely computationally expensive for complex real-world tasks. For example, tasks involving compute intensive object detectors as one of their components. In this work, we propose an approach that enables efficient large-scale testing using simplified low-fidelity simulators and without the computational cost of executing expensive deep learning models. Our approach relies on designing an efficient surrogate model corresponding to the compute intensive components of the task under test. We demonstrate the efficacy of our methodology by evaluating the performance of an autonomous driving task in the Carla simulator with reduced computational expense by training efficient surrogate models for PIXOR and CenterPoint LiDAR detectors, whilst demonstrating that the accuracy of the simulation is maintained.
△ Less
Submitted 4 November, 2021; v1 submitted 28 September, 2021;
originally announced October 2021.
-
Pretraining boosts out-of-domain robustness for pose estimation
Authors:
Alexander Mathis,
Thomas Biasi,
Steffen Schneider,
Mert Yüksekgönül,
Byron Rogers,
Matthias Bethge,
Mackenzie W. Mathis
Abstract:
Neural networks are highly effective tools for pose estimation. However, as in other computer vision tasks, robustness to out-of-domain data remains a challenge, especially for small training sets that are common for real-world applications. Here, we probe the generalization ability with three architecture classes (MobileNetV2s, ResNets, and EfficientNets) for pose estimation. We developed a datas…
▽ More
Neural networks are highly effective tools for pose estimation. However, as in other computer vision tasks, robustness to out-of-domain data remains a challenge, especially for small training sets that are common for real-world applications. Here, we probe the generalization ability with three architecture classes (MobileNetV2s, ResNets, and EfficientNets) for pose estimation. We developed a dataset of 30 horses that allowed for both "within-domain" and "out-of-domain" (unseen horse) benchmarking - this is a crucial test for robustness that current human pose estimation benchmarks do not directly address. We show that better ImageNet-performing architectures perform better on both within- and out-of-domain data if they are first pretrained on ImageNet. We additionally show that better ImageNet models generalize better across animal species. Furthermore, we introduce Horse-C, a new benchmark for common corruptions for pose estimation, and confirm that pretraining increases performance in this domain shift context as well. Overall, our results demonstrate that transfer learning is beneficial for out-of-domain robustness.
△ Less
Submitted 12 November, 2020; v1 submitted 24 September, 2019;
originally announced September 2019.
-
Inter-Scanner Harmonization of High Angular Resolution DW-MRI using Null Space Deep Learning
Authors:
Vishwesh Nath,
Prasanna Parvathaneni,
Colin B. Hansen,
Allison E. Hainline,
Camilo Bermudez,
Samuel Remedios,
Justin A. Blaber,
Kurt G. Schilling,
Ilwoo Lyu,
Vaibhav Janve,
Yurui Gao,
Iwona Stepniewska,
Baxter P. Rogers,
Allen T. Newton,
L. Taylor Davis,
Jeff Luci,
Adam W. Anderson,
Bennett A. Landman
Abstract:
Diffusion-weighted magnetic resonance imaging (DW-MRI) allows for non-invasive imaging of the local fiber architecture of the human brain at a millimetric scale. Multiple classical approaches have been proposed to detect both single (e.g., tensors) and multiple (e.g., constrained spherical deconvolution, CSD) fiber population orientations per voxel. However, existing techniques generally exhibit l…
▽ More
Diffusion-weighted magnetic resonance imaging (DW-MRI) allows for non-invasive imaging of the local fiber architecture of the human brain at a millimetric scale. Multiple classical approaches have been proposed to detect both single (e.g., tensors) and multiple (e.g., constrained spherical deconvolution, CSD) fiber population orientations per voxel. However, existing techniques generally exhibit low reproducibility across MRI scanners. Herein, we propose a data-driven tech-nique using a neural network design which exploits two categories of data. First, training data were acquired on three squirrel monkey brains using ex-vivo DW-MRI and histology of the brain. Second, repeated scans of human subjects were acquired on two different scanners to augment the learning of the network pro-posed. To use these data, we propose a new network architecture, the null space deep network (NSDN), to simultaneously learn on traditional observed/truth pairs (e.g., MRI-histology voxels) along with repeated observations without a known truth (e.g., scan-rescan MRI). The NSDN was tested on twenty percent of the histology voxels that were kept completely blind to the network. NSDN significantly improved absolute performance relative to histology by 3.87% over CSD and 1.42% over a recently proposed deep neural network approach. More-over, it improved reproducibility on the paired data by 21.19% over CSD and 10.09% over a recently proposed deep approach. Finally, NSDN improved gen-eralizability of the model to a third in vivo human scanner (which was not used in training) by 16.08% over CSD and 10.41% over a recently proposed deep learn-ing approach. This work suggests that data-driven approaches for local fiber re-construction are more reproducible, informative and precise and offers a novel, practical method for determining these models.
△ Less
Submitted 9 October, 2018;
originally announced October 2018.
-
Networks: An Economic Perspective
Authors:
Matthew O. Jackson,
Brian W. Rogers,
Yves Zenou
Abstract:
We discuss social network analysis from the perspective of economics. We organize the presentaion around the theme of externalities: the effects that one's behavior has on others' well-being. Externalities underlie the interdependencies that make networks interesting. We discuss network formation, as well as interactions between peoples' behaviors within a given network, and the implications in a…
▽ More
We discuss social network analysis from the perspective of economics. We organize the presentaion around the theme of externalities: the effects that one's behavior has on others' well-being. Externalities underlie the interdependencies that make networks interesting. We discuss network formation, as well as interactions between peoples' behaviors within a given network, and the implications in a variety of settings. Finally, we highlight some empirical challenges inherent in the statistical analysis of network-based data.
△ Less
Submitted 28 August, 2016;
originally announced August 2016.
-
Stochastic network formation and homophily
Authors:
Paolo Pin,
Brian Rogers
Abstract:
This is a chapter of the forthcoming Oxford Handbook on the Economics of Networks.
This is a chapter of the forthcoming Oxford Handbook on the Economics of Networks.
△ Less
Submitted 24 May, 2015;
originally announced May 2015.
-
Homophily and Long-Run Integration in Social Networks
Authors:
Yann Bramoullé,
Sergio Currarini,
Matthew O. Jackson,
Paolo Pin,
Brian W. Rogers
Abstract:
We model network formation when heterogeneous nodes enter sequentially and form connections through both random meetings and network-based search, but with type-dependent biases. We show that there is "long-run integration," whereby the composition of types in sufficiently old nodes' neighborhoods approaches the global type distribution, provided that the network-based search is unbiased. However,…
▽ More
We model network formation when heterogeneous nodes enter sequentially and form connections through both random meetings and network-based search, but with type-dependent biases. We show that there is "long-run integration," whereby the composition of types in sufficiently old nodes' neighborhoods approaches the global type distribution, provided that the network-based search is unbiased. However, younger nodes' connections still reflect the biased meetings process. We derive the type-based degree distributions and group-level homophily patterns when there are two types and location-based biases. Finally, we illustrate aspects of the model with an empirical application to data on citations in physics journals.
△ Less
Submitted 7 April, 2012; v1 submitted 22 January, 2012;
originally announced January 2012.
-
On the Stability of Swarm Consensus Under Noisy Control
Authors:
Gregory K. Fricke,
Bruce W. Rogers,
Devendra P. Garg
Abstract:
Representation of a swarm of independent robotic agents under graph-theoretic constructs allows for more formal analysis of convergence properties. We consider the local and global convergence behavior of an N-member swarm of agents in a modified consensus problem wherein the connectivity of agents is governed by probabilistic functions. The addition of a random walk control ensures Lyapunov stabi…
▽ More
Representation of a swarm of independent robotic agents under graph-theoretic constructs allows for more formal analysis of convergence properties. We consider the local and global convergence behavior of an N-member swarm of agents in a modified consensus problem wherein the connectivity of agents is governed by probabilistic functions. The addition of a random walk control ensures Lyapunov stability of the swarm consensus. Simulation results are given and planned experiments are described.
△ Less
Submitted 24 October, 2010;
originally announced October 2010.
