Showing 1–15 of 15 results for author: Rogers, S

Sea ice detection using concurrent multispectral and synthetic aperture radar imagery

Authors: Martin S J Rogers, Maria Fox, Andrew Fleming, Louisa van Zeeland, Jeremy Wilkinson, J. Scott Hosking

Abstract: Synthetic Aperture Radar (SAR) imagery is the primary data type used for sea ice map** due to its spatio-temporal coverage and the ability to detect sea ice independent of cloud and lighting conditions. Automatic sea ice detection using SAR imagery remains problematic due to the presence of ambiguous signal and noise within the image. Conversely, ice and water are easily distinguishable using mu… ▽ More Synthetic Aperture Radar (SAR) imagery is the primary data type used for sea ice map** due to its spatio-temporal coverage and the ability to detect sea ice independent of cloud and lighting conditions. Automatic sea ice detection using SAR imagery remains problematic due to the presence of ambiguous signal and noise within the image. Conversely, ice and water are easily distinguishable using multispectral imagery (MSI), but in the polar regions the ocean's surface is often occluded by cloud or the sun may not appear above the horizon for many months. To address some of these limitations, this paper proposes a new tool trained using concurrent multispectral Visible and SAR imagery for sea Ice Detection (ViSual\_IceD). ViSual\_IceD is a convolution neural network (CNN) that builds on the classic U-Net architecture by containing two parallel encoder stages, enabling the fusion and concatenation of MSI and SAR imagery containing different spatial resolutions. The performance of ViSual\_IceD is compared with U-Net models trained using concatenated MSI and SAR imagery as well as models trained exclusively on MSI or SAR imagery. ViSual\_IceD outperforms the other networks, with a F1 score 1.60\% points higher than the next best network, and results indicate that ViSual\_IceD is selective in the image type it uses during image segmentation. Outputs from ViSual\_IceD are compared to sea ice concentration products derived from the AMSR2 Passive Microwave (PMW) sensor. Results highlight how ViSual\_IceD is a useful tool to use in conjunction with PMW data, particularly in coastal regions. As the spatial-temporal coverage of MSI and SAR imagery continues to increase, ViSual\_IceD provides a new opportunity for robust, accurate sea ice coverage detection in polar regions. △ Less

Submitted 11 January, 2024; originally announced January 2024.

Comments: 34 pages, 10 figures, 2 tables

Improving Code Reviewer Recommendation: Accuracy, Latency, Workload, and Bystanders

Authors: Peter C. Rigby, Seth Rogers, Sadruddin Saleem, Parth Suresh, Daniel Suskin, Patrick Riggs, Chandra Maddila, Nachiappan Nagappan

Abstract: Code review ensures that a peer engineer manually examines the code before it is integrated and released into production. At Meta, we develop a wide range of software at scale, from social networking to software development infrastructure, such as calendar and meeting tools to continuous integration. We are constantly improving our code review system, and in this work we describe a series of exper… ▽ More Code review ensures that a peer engineer manually examines the code before it is integrated and released into production. At Meta, we develop a wide range of software at scale, from social networking to software development infrastructure, such as calendar and meeting tools to continuous integration. We are constantly improving our code review system, and in this work we describe a series of experiments that were conducted across 10's of thousands of engineers and 100's of thousands of reviews. We build upon the recommender that has been in production since 2018, RevRecV1. We found that reviewers were being assigned based on prior authorship of files. We reviewed the literature for successful features and experimented with them with RevRecV2 in production. The most important feature in our new model was the familiarity of the author and reviewer, we saw an overall improvement in accuracy of 14 percentage points. Prior research has shown that reviewer workload is skewed. To balance workload, we divide the reviewer score from RevRecV2 by each candidate reviewers workload. We experimented with multiple types of workload to develop RevRecWL. We find that reranking candidate reviewers by workload often leads to a reviewers with lower workload being selected by authors. The bystander effect can occur when a team of reviewers is assigned the review. We mitigate the bystander effect by randomly assigning one of the recommended reviewers. Having an individual who is responsible for the review, reduces the time take for reviews by -11%. △ Less

Submitted 28 December, 2023; originally announced December 2023.

An assessment of measuring local levels of homelessness through proxy social media signals

Authors: Yoshi Meke Bird, Sarah E. Grobe, Michael V. Arnold, Sean P. Rogers, Mikaela I. Fudolig, Julia Witte Zimmerman, Christopher M. Danforth, Peter Sheridan Dodds

Abstract: Recent studies suggest social media activity can function as a proxy for measures of state-level public health, detectable through natural language processing. We present results of our efforts to apply this approach to estimate homelessness at the state level throughout the US during the period 2010-2019 and 2022 using a dataset of roughly 1 million geotagged tweets containing the substring ``hom… ▽ More Recent studies suggest social media activity can function as a proxy for measures of state-level public health, detectable through natural language processing. We present results of our efforts to apply this approach to estimate homelessness at the state level throughout the US during the period 2010-2019 and 2022 using a dataset of roughly 1 million geotagged tweets containing the substring ``homeless.'' Correlations between homelessness-related tweet counts and ranked per capita homelessness volume, but not general-population densities, suggest a relationship between the likelihood of Twitter users to personally encounter or observe homelessness in their everyday lives and their likelihood to communicate about it online. An increase to the log-odds of ``homeless'' appearing in an English-language tweet, as well as an acceleration in the increase in average tweet sentiment, suggest that tweets about homelessness are also affected by trends at the nation-scale. Additionally, changes to the lexical content of tweets over time suggest that reversals to the polarity of national or state-level trends may be detectable through an increase in political or service-sector language over the semantics of charity or direct appeals. An analysis of user account type also revealed changes to Twitter-use patterns by accounts authored by individuals versus entities that may provide an additional signal to confirm changes to homelessness density in a given jurisdiction. While a computational approach to social media analysis may provide a low-cost, real-time dataset rich with information about nationwide and localized impacts of homelessness and homelessness policy, we find that practical issues abound, limiting the potential of social media as a proxy to complement other measures of homelessness. △ Less

Submitted 15 May, 2023; originally announced May 2023.

Comments: 29 pages, 21 figures

Representational Tenets for Memory Athletics

Authors: Kevin Schmidt, Othalia Larue, Ray Kulhanek, Dylan Flaute, Razvan Veliche, Christian Manasseh, Nelson Dellis, Scott Clouse, Jared Culbertson, Steve Rogers

Abstract: We describe the current state of world-class memory competitions, including the methods used to prepare for and compete in memory competitions, based on the subjective report of World Memory Championship Grandmaster and co-author Nelson Dellis. We then explore the reported experiences through the lens of the Simulated, Situated, and Structurally coherent Qualia (S3Q) theory of consciousness, in or… ▽ More We describe the current state of world-class memory competitions, including the methods used to prepare for and compete in memory competitions, based on the subjective report of World Memory Championship Grandmaster and co-author Nelson Dellis. We then explore the reported experiences through the lens of the Simulated, Situated, and Structurally coherent Qualia (S3Q) theory of consciousness, in order to propose a set of experiments to help further understand the boundaries of expert memory performance. △ Less

Submitted 22 February, 2023; originally announced March 2023.

Safe machine learning model release from Trusted Research Environments: The AI-SDC package

Authors: Jim Smith, Richard J. Preen, Andrew McCarthy, Alba Crespi-Boixader, James Liley, Simon Rogers

Abstract: We present AI-SDC, an integrated suite of open source Python tools to facilitate Statistical Disclosure Control (SDC) of Machine Learning (ML) models trained on confidential data prior to public release. AI-SDC combines (i) a SafeModel package that extends commonly used ML models to provide ante-hoc SDC by assessing the vulnerability of disclosure posed by the training regime; and (ii) an Attacks… ▽ More We present AI-SDC, an integrated suite of open source Python tools to facilitate Statistical Disclosure Control (SDC) of Machine Learning (ML) models trained on confidential data prior to public release. AI-SDC combines (i) a SafeModel package that extends commonly used ML models to provide ante-hoc SDC by assessing the vulnerability of disclosure posed by the training regime; and (ii) an Attacks package that provides post-hoc SDC by rigorously assessing the empirical disclosure risk of a model through a variety of simulated attacks after training. The AI-SDC code and documentation are available under an MIT license at https://github.com/AI-SDC/AI-SDC. △ Less

Submitted 6 December, 2022; v1 submitted 2 December, 2022; originally announced December 2022.

GRAIMATTER Green Paper: Recommendations for disclosure control of trained Machine Learning (ML) models from Trusted Research Environments (TREs)

Authors: Emily Jefferson, James Liley, Maeve Malone, Smarti Reel, Alba Crespi-Boixader, Xaroula Kerasidou, Francesco Tava, Andrew McCarthy, Richard Preen, Alberto Blanco-Justicia, Esma Mansouri-Benssassi, Josep Domingo-Ferrer, Jillian Beggs, Antony Chuter, Christian Cole, Felix Ritchie, Angela Daly, Simon Rogers, Jim Smith

Abstract: TREs are widely, and increasingly used to support statistical analysis of sensitive data across a range of sectors (e.g., health, police, tax and education) as they enable secure and transparent research whilst protecting data confidentiality. There is an increasing desire from academia and industry to train AI models in TREs. The field of AI is develo** quickly with applications including spott… ▽ More TREs are widely, and increasingly used to support statistical analysis of sensitive data across a range of sectors (e.g., health, police, tax and education) as they enable secure and transparent research whilst protecting data confidentiality. There is an increasing desire from academia and industry to train AI models in TREs. The field of AI is develo** quickly with applications including spotting human errors, streamlining processes, task automation and decision support. These complex AI models require more information to describe and reproduce, increasing the possibility that sensitive personal data can be inferred from such descriptions. TREs do not have mature processes and controls against these risks. This is a complex topic, and it is unreasonable to expect all TREs to be aware of all risks or that TRE researchers have addressed these risks in AI-specific training. GRAIMATTER has developed a draft set of usable recommendations for TREs to guard against the additional risks when disclosing trained AI models from TREs. The development of these recommendations has been funded by the GRAIMATTER UKRI DARE UK sprint research project. This version of our recommendations was published at the end of the project in September 2022. During the course of the project, we have identified many areas for future investigations to expand and test these recommendations in practice. Therefore, we expect that this document will evolve over time. △ Less

Submitted 3 November, 2022; originally announced November 2022.

Diverse Misinformation: Impacts of Human Biases on Detection of Deepfakes on Networks

Authors: Juniper Lovato, Laurent Hébert-Dufresne, Jonathan St-Onge, Randall Harp, Gabriela Salazar Lopez, Sean P. Rogers, Ijaz Ul Haq, Jeremiah Onaolapo

Abstract: Social media platforms often assume that users can self-correct against misinformation. However, social media users are not equally susceptible to all misinformation as their biases influence what types of misinformation might thrive and who might be at risk. We call "diverse misinformation" the complex relationships between human biases and demographics represented in misinformation. To investiga… ▽ More Social media platforms often assume that users can self-correct against misinformation. However, social media users are not equally susceptible to all misinformation as their biases influence what types of misinformation might thrive and who might be at risk. We call "diverse misinformation" the complex relationships between human biases and demographics represented in misinformation. To investigate how users' biases impact their susceptibility and their ability to correct each other, we analyze classification of deepfakes as a type of diverse misinformation. We chose deepfakes as a case study for three reasons: 1) their classification as misinformation is more objective; 2) we can control the demographics of the personas presented; 3) deepfakes are a real-world concern with associated harms that must be better understood. Our paper presents an observational survey (N=2,016) where participants are exposed to videos and asked questions about their attributes, not knowing some might be deepfakes. Our analysis investigates the extent to which different users are duped and which perceived demographics of deepfake personas tend to mislead. We find that accuracy varies by demographics, and participants are generally better at classifying videos that match them. We extrapolate from these results to understand the potential population-level impacts of these biases using a mathematical model of the interplay between diverse misinformation and crowd correction. Our model suggests that diverse contacts might provide "herd correction" where friends can protect each other. Altogether, human biases and the attributes of misinformation matter greatly, but having a diverse social group may help reduce susceptibility to misinformation. △ Less

Submitted 13 January, 2024; v1 submitted 18 October, 2022; originally announced October 2022.

Comments: Supplementary appendix available upon request for the time being

Machine Learning Models Disclosure from Trusted Research Environments (TRE), Challenges and Opportunities

Authors: Esma Mansouri-Benssassi, Simon Rogers, Jim Smith, Felix Ritchie, Emily Jefferson

Abstract: Artificial intelligence (AI) applications in healthcare and medicine have increased in recent years. To enable access to personal data, Trusted Research environments (TREs) provide safe and secure environments in which researchers can access sensitive personal data and develop Artificial Intelligence (AI) and Machine Learning models. However currently few TREs support the use of automated AI-based… ▽ More Artificial intelligence (AI) applications in healthcare and medicine have increased in recent years. To enable access to personal data, Trusted Research environments (TREs) provide safe and secure environments in which researchers can access sensitive personal data and develop Artificial Intelligence (AI) and Machine Learning models. However currently few TREs support the use of automated AI-based modelling using Machine Learning. Early attempts have been made in the literature to present and introduce privacy preserving machine learning from the design point of view [1]. However, there exists a gap in the practical decision-making guidance for TREs in handling models disclosure. Specifically, the use of machine learning creates a need to disclose new types of outputs from TREs, such as trained machine learning models. Although TREs have clear policies for the disclosure of statistical outputs, the extent to which trained models can leak personal training data once released is not well understood and guidelines do not exist within TREs for the safe disclosure of these models. In this paper we introduce the challenge of disclosing trained machine learning models from TREs. We first give an overview of machine learning models in general and describe some of their applications in healthcare and medicine. We define the main vulnerabilities of trained machine learning models in general. We also describe the main factors affecting the vulnerabilities of disclosing machine learning models. This paper also provides insights and analyses methods that could be introduced within TREs to mitigate the risk of privacy breaches when disclosing trained models. △ Less

Submitted 20 August, 2022; v1 submitted 10 November, 2021; originally announced November 2021.

Forward and Inverse models in HCI:Physical simulation and deep learning for inferring 3D finger pose

Authors: Roderick Murray-Smith, John H. Williamson, Andrew Ramsay, Francesco Tonolini, Simon Rogers, Antoine Loriette

Abstract: We outline the role of forward and inverse modelling approaches in the design of human--computer interaction systems. Causal, forward models tend to be easier to specify and simulate, but HCI requires solutions of the inverse problem. We infer finger 3D position $(x,y,z)$ and pose (pitch and yaw) on a mobile device using capacitive sensors which can sense the finger up to 5cm above the screen. We… ▽ More We outline the role of forward and inverse modelling approaches in the design of human--computer interaction systems. Causal, forward models tend to be easier to specify and simulate, but HCI requires solutions of the inverse problem. We infer finger 3D position $(x,y,z)$ and pose (pitch and yaw) on a mobile device using capacitive sensors which can sense the finger up to 5cm above the screen. We use machine learning to develop data-driven models to infer position, pose and sensor readings, based on training data from: 1. data generated by robots, 2. data from electrostatic simulators 3. human-generated data. Machine learned emulation is used to accelerate the electrostatic simulation performance by a factor of millions. We combine a Conditional Variational Autoencoder with domain expertise/models experimentally collected data. We compare forward and inverse model approaches to direct inference of finger pose. The combination gives the most accurate reported results on inferring 3D position and pose with a capacitive sensor on a mobile device. △ Less

Submitted 7 September, 2021; originally announced September 2021.

REVAMP$^2$T: Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking

Authors: Christopher Neff, Matías Mendieta, Shrey Mohan, Mohammadreza Baharani, Samuel Rogers, Hamed Tabkhi

Abstract: This article presents REVAMP$^2$T, Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking, as an integrated end-to-end IoT system for privacy-built-in decentralized situational awareness. REVAMP$^2$T presents novel algorithmic and system constructs to push deep learning and video analytics next to IoT devices (i.e. video cameras). On the algorithm side, REVAMP$^2$T propo… ▽ More This article presents REVAMP$^2$T, Real-time Edge Video Analytics for Multi-camera Privacy-aware Pedestrian Tracking, as an integrated end-to-end IoT system for privacy-built-in decentralized situational awareness. REVAMP$^2$T presents novel algorithmic and system constructs to push deep learning and video analytics next to IoT devices (i.e. video cameras). On the algorithm side, REVAMP$^2$T proposes a unified integrated computer vision pipeline for detection, re-identification, and tracking across multiple cameras without the need for storing the streaming data. At the same time, it avoids facial recognition, and tracks and re-identifies pedestrians based on their key features at runtime. On the IoT system side, REVAMP$^2$T provides infrastructure to maximize hardware utilization on the edge, orchestrates global communications, and provides system-wide re-identification, without the use of personally identifiable information, for a distributed IoT network. For the results and evaluation, this article also proposes a new metric, Accuracy$\cdot$Efficiency (Æ), for holistic evaluation of IoT systems for real-time video analytics based on accuracy, performance, and power efficiency. REVAMP$^2$T outperforms current state-of-the-art by as much as thirteen-fold Æ~improvement. △ Less

Submitted 25 November, 2019; v1 submitted 20 November, 2019; originally announced November 2019.

Comments: Published as an article paper in IEEE Internet of Things Journal: Special Issue on Privacy and Security in Distributed Edge Computing and Evolving IoT

SIG-DB: leveraging homomorphic encryption to Securely Interrogate privately held Genomic DataBases

Authors: Alexander J. Titus, Audrey Flower, Patrick Hagerty, Paul Gamble, Charlie Lewis, Todd Stavish, Kevin P. OConnell, Greg Shipley, Stephanie M. Rogers

Abstract: Genomic data are becoming increasingly valuable as we develop methods to utilize the information at scale and gain a greater understanding of how genetic information relates to biological function. Advances in synthetic biology and the decreased cost of sequencing are increasing the amount of privately held genomic data. As the quantity and value of private genomic data grows, so does the incentiv… ▽ More Genomic data are becoming increasingly valuable as we develop methods to utilize the information at scale and gain a greater understanding of how genetic information relates to biological function. Advances in synthetic biology and the decreased cost of sequencing are increasing the amount of privately held genomic data. As the quantity and value of private genomic data grows, so does the incentive to acquire and protect such data, which creates a need to store and process these data securely. We present an algorithm for the Secure Interrogation of Genomic DataBases (SIG-DB). The SIG-DB algorithm enables databases of genomic sequences to be searched with an encrypted query sequence without revealing the query sequence to the Database Owner or any of the database sequences to the Querier. SIG-DB is the first application of its kind to take advantage of locality-sensitive hashing and homomorphic encryption to allow generalized sequence-to-sequence comparisons of genomic data. △ Less

Submitted 26 March, 2018; originally announced March 2018.

Comments: 38 pages, 3 figures, 4 tables, 1 supplemental table, 7 supplemental figures

Report number: PMID: 30180163

Journal ref: PLoS Computational Biology; 2018 Sep 4; 14(9):e1006454

PySEAL: A Python wrapper implementation of the SEAL homomorphic encryption library

Authors: Alexander J. Titus, Shashwat Kishore, Todd Stavish, Stephanie M. Rogers, Karl Ni

Abstract: Motivation: The ability to perform operations on encrypted data has a growing number of applications in bioinformatics, with implications for data privacy in health care and biosecurity. The SEAL library is a popular implementation of fully homomorphic encryption developed in C++ by Microsoft Research. Despite the advantages of C++, Python is a flexible and dominant programming language that enabl… ▽ More Motivation: The ability to perform operations on encrypted data has a growing number of applications in bioinformatics, with implications for data privacy in health care and biosecurity. The SEAL library is a popular implementation of fully homomorphic encryption developed in C++ by Microsoft Research. Despite the advantages of C++, Python is a flexible and dominant programming language that enables rapid prototy** of bioinformatics pipelines. Results: In an effort to make homomorphic encryption accessible to a broader range of bioinformatics scientists and applications, we present a Python binding implementation of the popular homomorphic encryption library, SEAL, using pybind11. The software contains a Docker image to facilitate easy installation and execution of the SEAL build process. Availability: All code is publicly available at https://github.com/Lab41/PySEAL Contact: [email protected] Supplementary information: Supplementary information is available on the Lab41 GitHub. △ Less

Submitted 5 March, 2018; originally announced March 2018.

Comments: 2 pages, 1 figure

Single-Shot Clothing Category Recognition in Free-Configurations with Application to Autonomous Clothes Sorting

Authors: Li Sun, Gerardo Aragon-Camarasa, Simon Rogers, Rustam Stolkin, J. Paul Siebert

Abstract: This paper proposes a single-shot approach for recognising clothing categories from 2.5D features. We propose two visual features, BSP (B-Spline Patch) and TSD (Topology Spatial Distances) for this task. The local BSP features are encoded by LLC (Locality-constrained Linear Coding) and fused with three different global features. Our visual feature is robust to deformable shapes and our approach is… ▽ More This paper proposes a single-shot approach for recognising clothing categories from 2.5D features. We propose two visual features, BSP (B-Spline Patch) and TSD (Topology Spatial Distances) for this task. The local BSP features are encoded by LLC (Locality-constrained Linear Coding) and fused with three different global features. Our visual feature is robust to deformable shapes and our approach is able to recognise the category of unknown clothing in unconstrained and random configurations. We integrated the category recognition pipeline with a stereo vision system, clothing instance detection, and dual-arm manipulators to achieve an autonomous sorting system. To verify the performance of our proposed method, we build a high-resolution RGBD clothing dataset of 50 clothing items of 5 categories sampled in random configurations (a total of 2,100 clothing samples). Experimental results show that our approach is able to reach 83.2\% accuracy while classifying clothing items which were previously unseen during training. This advances beyond the previous state-of-the-art by 36.2\%. Finally, we evaluate the proposed approach in an autonomous robot sorting system, in which the robot recognises a clothing item from an unconstrained pile, grasps it, and sorts it into a box according to its category. Our proposed sorting system achieves reasonable sorting success rates with single-shot perception. △ Less

Submitted 22 July, 2017; originally announced July 2017.

Comments: 9 pages, accepted by IROS2017

Related Pins at Pinterest: The Evolution of a Real-World Recommender System

Authors: David C. Liu, Stephanie Rogers, Raymond Shiau, Dmitry Kislyuk, Kevin C. Ma, Zhigang Zhong, Jenny Liu, Yushi **g

Abstract: Related Pins is the Web-scale recommender system that powers over 40% of user engagement on Pinterest. This paper is a longitudinal study of three years of its development, exploring the evolution of the system and its components from prototypes to present state. Each component was originally built with many constraints on engineering effort and computational resources, so we prioritized the simpl… ▽ More Related Pins is the Web-scale recommender system that powers over 40% of user engagement on Pinterest. This paper is a longitudinal study of three years of its development, exploring the evolution of the system and its components from prototypes to present state. Each component was originally built with many constraints on engineering effort and computational resources, so we prioritized the simplest and highest-leverage solutions. We show how organic growth led to a complex system and how we managed this complexity. Many challenges arose while building this system, such as avoiding feedback loops, evaluating performance, activating content, and eliminating legacy heuristics. Finally, we offer suggestions for tackling these challenges when engineering Web-scale recommender systems. △ Less

Submitted 25 February, 2017; originally announced February 2017.

Robot Vision Architecture for Autonomous Clothes Manipulation

Authors: Li Sun, Gerardo Aragon-Camarasa, Simon Rogers, J. Paul Siebert

Abstract: This paper presents a novel robot vision architecture for perceiving generic 3D clothes configurations. Our architecture is hierarchically structured, starting from low-level curvatures, across mid-level geometric shapes \& topology descriptions; and finally approaching high-level semantic surface structure descriptions. We demonstrate our robot vision architecture in a customised dual-arm industr… ▽ More This paper presents a novel robot vision architecture for perceiving generic 3D clothes configurations. Our architecture is hierarchically structured, starting from low-level curvatures, across mid-level geometric shapes \& topology descriptions; and finally approaching high-level semantic surface structure descriptions. We demonstrate our robot vision architecture in a customised dual-arm industrial robot with our self-designed, off-the-self stereo vision system, carrying out autonomous gras** and dual-arm flattening. It is worth noting that the proposed dual-arm flattening approach is unique among the state-of-the-art robot autonomous system, which is the major contribution of this paper. The experimental results show that the proposed dual-arm flattening using stereo vision system remarkably outperforms the single-arm flattening and widely-cited Kinect-based sensing system for dexterous manipulation tasks. In addition, the proposed gras** approach achieves satisfactory performance on gras** various kind of garments, verifying the capability of proposed visual perception architecture to be adapted to more than one clothing manipulation tasks. △ Less

Submitted 18 October, 2016; originally announced October 2016.

Comments: 14 pages, under review