Showing 1–50 of 137 results for author: Nichols, J

Bending Light via Transverse Momentum Exchange: Theory and Experiment

Authors: J. M. Nichols, D. V. Nickel, F. Bucholtz

Abstract: In this work we present a transport model for the paraxial propagation of vector beams. Of particular importance is the appearance of a new transverse momentum term proportional to the linear polarization angle gradient imparted to the beam during the beam preparation process. The model predicts that during propagation, this new momentum will be exchanged for classical transverse momentum, thereby… ▽ More In this work we present a transport model for the paraxial propagation of vector beams. Of particular importance is the appearance of a new transverse momentum term proportional to the linear polarization angle gradient imparted to the beam during the beam preparation process. The model predicts that during propagation, this new momentum will be exchanged for classical transverse momentum, thereby causing the beam to accelerate. We observe the momentum exchange in experiment by designing the polarization profile in such a way as to cause the beam centroid to follow a parabolic path through free space. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Comments: 11 pages, 1 figure, 3 appendices

MSC Class: 35Q60; 78A60

UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback

Authors: Jason Wu, Eldon Schoop, Alan Leung, Titus Barik, Jeffrey P. Bigham, Jeffrey Nichols

Abstract: Large language models (LLMs) struggle to consistently generate UI code that compiles and produces visually relevant designs. Existing approaches to improve generation rely on expensive human feedback or distilling a proprietary model. In this paper, we explore the use of automated feedback (compilers and multi-modal models) to guide LLMs to generate high-quality UI code. Our method starts with an… ▽ More Large language models (LLMs) struggle to consistently generate UI code that compiles and produces visually relevant designs. Existing approaches to improve generation rely on expensive human feedback or distilling a proprietary model. In this paper, we explore the use of automated feedback (compilers and multi-modal models) to guide LLMs to generate high-quality UI code. Our method starts with an existing LLM and iteratively produces improved models by self-generating a large synthetic dataset using an original model, applying automated tools to aggressively filter, score, and de-duplicate the data into a refined higher quality dataset. The original LLM is improved by finetuning on this refined dataset. We applied our approach to several open-source LLMs and compared the resulting performance to baseline models with both automated metrics and human preferences. Our evaluation shows the resulting models outperform all other downloadable baselines and approach the performance of larger proprietary models. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: Accepted to NAACL 2024

X-ray and UV radiation in the planet-forming T-Tauri system PDS 70. Signs of accretion and coronal activity

Authors: Simon R. G. Joyce, John P. Pye, Jonathan D. Nichols, Richard Alexander, Manuel Gudel, David Barrado

Abstract: Planet formation takes place in protoplanetary discs around young T-Tauri stars. PDS 70 is one of the first confirmed examples of a system where the planets are currently forming in gaps in the disc, and can be directly imaged. One of the main early influences on planet formation is the lifetime of the protoplanetary disk, which is limited by the intense stellar X-ray and UV radiation. Stellar cor… ▽ More Planet formation takes place in protoplanetary discs around young T-Tauri stars. PDS 70 is one of the first confirmed examples of a system where the planets are currently forming in gaps in the disc, and can be directly imaged. One of the main early influences on planet formation is the lifetime of the protoplanetary disk, which is limited by the intense stellar X-ray and UV radiation. Stellar coronal activity and accretion of material onto the star are both potential sources of XUV radiation. Previous \textit{Swift} observations detected UV emission, which were consistent with a low rate of accretion. We present follow up observations with the XMM-Newton observatory, which observed PDS 70 simultaneously in X-ray and UV in order to determine intensity of XUV radiation in the system, and identify if the source is coronal, accretion, or both. We detect a strong source in both X-ray and UV, with an average X-ray 0.2-12 keV luminosity of $1.37\times10^{30}\ \mathrm{erg\ s}^{-1}$, and a possible flare which increased the luminosity to $2.8\times10^{30}\ \mathrm{erg\ s}^{-1}$. The UV flux density is in excess of what would be expected from chromospheric emission, and supports the interpretation that PDS 70 has continuing weak accretion less than $\sim10^{-10}\ \mathrm{M_{\odot}\ yr^{-1}}$. The implications of the detected X-ray and UV radiation are that the disc is likely to be in the final stages of dispersal, and will be completely evaporated in the next million years, bringing an end to the primary planet formation process. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Comments: 17 pages, Published in MNRAS

Journal ref: MNRAS (2023), 519, 4514

ExoplANETS-A: A VO database for host stars and planetary systems: The effect of XUV on planet atmospheres

Authors: M. Morales-Calderón, S. R. G. Joyce, J. P. Pye, D. Barrado, M. García Castro, C. Rodrigo, E. Solano, J. D. Nichols, P. O. Lagage, A. Castro-González, R. A. García, M. Guedel, N. Huélamo, Y. Metodieva, R. Waters

Abstract: ExoplANETS-A is an EU Horizon-2020 project with the primary objective of establishing new knowledge on exoplanet atmospheres. Intimately related to this topic is the study of the host-stars radiative properties in order to understand the environment in which exoplanets lie. The aim of this work is to exploit archived data from space-based observatories and other public sources to produce uniform… ▽ More ExoplANETS-A is an EU Horizon-2020 project with the primary objective of establishing new knowledge on exoplanet atmospheres. Intimately related to this topic is the study of the host-stars radiative properties in order to understand the environment in which exoplanets lie. The aim of this work is to exploit archived data from space-based observatories and other public sources to produce uniform sets of stellar data that can establish new insight on the influence of the host star on the planetary atmosphere. We have compiled X-ray and UV luminosities, which affect the formation and the atmospheric properties of the planets, and stellar parameters, which impact the retrieval process of the planetary-atmosphere's properties and its errors. Our sample is formed of all transiting-exoplanet systems observed by HST or Spitzer. It includes 205 exoplanets and their 114 host-stars. We have built a catalogue with information extracted from public, online archives augmented by quantities derived by the Exoplanets-A work. With this catalogue we have implemented an online database which also includes X-ray and OHP spectra and TESS light curves. In addition, we have developed a tool, exoVOSA, which is able to fit the spectral energy distribution of exoplanets. We give an example of using the database to study the effects of the host-star high-energy emission on the exoplanet atmosphere. The sample has a planet radius valley which is located at 1.8 Earth radii, in agreement with previous studies. Multiplanet systems in our sample were used to test the photoevaporation model and we find that out of 14 systems, only one significant case poses a contradiction to it (K2-3). In summary, the exoplanet and stellar resources compiled and generated by ExoplANETS-A form a sound basis for current JWST observations and for future work in the era of Ariel. △ Less

Submitted 10 May, 2024; originally announced May 2024.

Comments: 14 pages

The Nature of X-Rays from Young Stellar Objects in the Orion Nebula Cluster -- A Chandra HETGS Legacy Project

Authors: Norbert S. Schulz, David P. Huenemoerder, David A. Principe, Marc Gagne, Hans Moritz Günther, Joel Kastner, Joy Nichols, Andrew Pollock, Thomas Preibisch, Paola Testa, Fabio Reale, Fabio Favata, Claude R. Canizares

Abstract: The Orion Nebula Cluster (ONC) is the closest site of very young ($\sim$ 1 Myrs) massive star formation. The ONC hosts more than 1600 young and X-ray bright stars with masses ranging from $\sim$ 0.1 to 35 $M_\odot$. The Chandra HETGS Orion Legacy Project observed the ONC with the Chandra high energy transmission grating spectrometer (HETGS) for $2.1\,$Ms. We describe the spectral extraction and cl… ▽ More The Orion Nebula Cluster (ONC) is the closest site of very young ($\sim$ 1 Myrs) massive star formation. The ONC hosts more than 1600 young and X-ray bright stars with masses ranging from $\sim$ 0.1 to 35 $M_\odot$. The Chandra HETGS Orion Legacy Project observed the ONC with the Chandra high energy transmission grating spectrometer (HETGS) for $2.1\,$Ms. We describe the spectral extraction and cleaning processes necessary to separate overlap** spectra. We obtained 36 high resolution spectra which includes a high brilliance X-ray spectrum of $θ^1$ Ori C with over 100 highly significant X-ray lines. The lines show Doppler broadening between 300 and $400\;\mathrm{km}\;\mathrm{s}^{-1}$. Higher spectral diffraction orders allow us to resolve line components of high Z He-like triplets in $θ^1$ Ori C with unprecedented spectral resolution. Long term light curves spanning $\sim$20 years show all stars to be highly variable, including the massive stars. Spectral fitting with thermal coronal emission line models reveals that most sources show column densities of up to a few times $10^{22}\,$cm$^{-2}$ and high coronal temperatures of 10 to 90 MK. We observe a bifurcation of the high temperature component where some stars show a high component of 40 MK, while others show above 60 MK indicating heavy flaring activity. Some lines are resolved with Doppler broadening above our threshold of $\sim200\;\mathrm{km}\;\mathrm{s}^{-1}$, up to $500\;\mathrm{km}\;\mathrm{s}^{-1}$. This data set represents the largest collection of HETGS high resolution X-ray spectra from young pre-MS stars in a single star-forming region to date. △ Less

Submitted 9 May, 2024; v1 submitted 30 April, 2024; originally announced April 2024.

Comments: Accepted by ApJ, fixed typo in the xlabel of Fig 12 (left panel)

Chandra HETG X-ray Spectra and Variability of $π$ Aqr, a $γ$ Cas-type Be star

Authors: David P. Huenemoerder, Pragati Pradhan, Claude R. Canizares, Sean Gunderson, Richard Ignace, Joy S. Nichols, A. M. T. Pollock, Norbert S. Schulz, Dustin K. Swarm, Jose M. Torrejon

Abstract: High-resolution X-ray spectra of $π\,$Aqr, a $γ\,$Cas-type star, obtained with the Chandra/HETG grating spectrometer, revealed emission lines of H-like ions of Mg, Si, S, and Fe, a strong, hard continuum, and a lack of He-like ions, indicating the presence of very hot thermal plasma. The X-ray light curve showed significant fluctuations, with coherent variability at period of about 3400 seconds in… ▽ More High-resolution X-ray spectra of $π\,$Aqr, a $γ\,$Cas-type star, obtained with the Chandra/HETG grating spectrometer, revealed emission lines of H-like ions of Mg, Si, S, and Fe, a strong, hard continuum, and a lack of He-like ions, indicating the presence of very hot thermal plasma. The X-ray light curve showed significant fluctuations, with coherent variability at period of about 3400 seconds in one observation. The hardness ratio was relatively constant except for one observation in which the spectrum was much harder and more absorbed. We interpret the X-ray emission as arising from accretion onto the secondary, which is likely a magnetic white dwarf, an intermediate polar system. △ Less

Submitted 25 April, 2024; originally announced April 2024.

Comments: 7 pages, 3 figures, 3 tables

UIClip: A Data-driven Model for Assessing User Interface Design

Authors: Jason Wu, Yi-Hao Peng, Amanda Li, Amanda Swearngin, Jeffrey P. Bigham, Jeffrey Nichols

Abstract: User interface (UI) design is a difficult yet important task for ensuring the usability, accessibility, and aesthetic qualities of applications. In our paper, we develop a machine-learned model, UIClip, for assessing the design quality and visual relevance of a UI given its screenshot and natural language description. To train UIClip, we used a combination of automated crawling, synthetic augmenta… ▽ More User interface (UI) design is a difficult yet important task for ensuring the usability, accessibility, and aesthetic qualities of applications. In our paper, we develop a machine-learned model, UIClip, for assessing the design quality and visual relevance of a UI given its screenshot and natural language description. To train UIClip, we used a combination of automated crawling, synthetic augmentation, and human ratings to construct a large-scale dataset of UIs, collated by description and ranked by design quality. Through training on the dataset, UIClip implicitly learns properties of good and bad designs by i) assigning a numerical score that represents a UI design's relevance and quality and ii) providing design suggestions. In an evaluation that compared the outputs of UIClip and other baselines to UIs rated by 12 human designers, we found that UIClip achieved the highest agreement with ground-truth rankings. Finally, we present three example applications that demonstrate how UIClip can facilitate downstream applications that rely on instantaneous assessment of UI design quality: i) UI code generation, ii) UI design tips generation, and iii) quality-aware UI example search. △ Less

Submitted 18 April, 2024; originally announced April 2024.

BISCUIT: Scaffolding LLM-Generated Code with Ephemeral UIs in Computational Notebooks

Authors: Ruijia Cheng, Titus Barik, Alan Leung, Fred Hohman, Jeffrey Nichols

Abstract: Novices frequently engage with machine learning tutorials in computational notebooks and have been adopting code generation technologies based on large language models (LLMs). However, they encounter difficulties in understanding and working with code produced by LLMs. To mitigate these challenges, we introduce a novel workflow into computational notebooks that augments LLM-based code generation w… ▽ More Novices frequently engage with machine learning tutorials in computational notebooks and have been adopting code generation technologies based on large language models (LLMs). However, they encounter difficulties in understanding and working with code produced by LLMs. To mitigate these challenges, we introduce a novel workflow into computational notebooks that augments LLM-based code generation with an additional ephemeral UI step, offering users UI-based scaffolds as an intermediate stage between user prompts and code generation. We present this workflow in BISCUIT, an extension for JupyterLab that provides users with ephemeral UIs generated by LLMs based on the context of their code and intentions, scaffolding users to understand, guide, and explore with LLM-generated code. Through a user study where 10 novices used BISCUIT for machine learning tutorials, we discover that BISCUIT offers user semantic representation of code to aid their understanding, reduces the complexity of prompt engineering, and creates a playground for users to explore different variables and iterate on their ideas. We discuss the implications of our findings for UI-centric interactive paradigm in code generation LLMs. △ Less

Submitted 12 April, 2024; v1 submitted 10 April, 2024; originally announced April 2024.

Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs

Authors: Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang, Zhe Gan

Abstract: Recent advancements in multimodal large language models (MLLMs) have been noteworthy, yet, these general-domain MLLMs often fall short in their ability to comprehend and interact effectively with user interface (UI) screens. In this paper, we present Ferret-UI, a new MLLM tailored for enhanced understanding of mobile UI screens, equipped with referring, grounding, and reasoning capabilities. Given… ▽ More Recent advancements in multimodal large language models (MLLMs) have been noteworthy, yet, these general-domain MLLMs often fall short in their ability to comprehend and interact effectively with user interface (UI) screens. In this paper, we present Ferret-UI, a new MLLM tailored for enhanced understanding of mobile UI screens, equipped with referring, grounding, and reasoning capabilities. Given that UI screens typically exhibit a more elongated aspect ratio and contain smaller objects of interest (e.g., icons, texts) than natural images, we incorporate "any resolution" on top of Ferret to magnify details and leverage enhanced visual features. Specifically, each screen is divided into 2 sub-images based on the original aspect ratio (i.e., horizontal division for portrait screens and vertical division for landscape screens). Both sub-images are encoded separately before being sent to LLMs. We meticulously gather training samples from an extensive range of elementary UI tasks, such as icon recognition, find text, and widget listing. These samples are formatted for instruction-following with region annotations to facilitate precise referring and grounding. To augment the model's reasoning ability, we further compile a dataset for advanced tasks, including detailed description, perception/interaction conversations, and function inference. After training on the curated datasets, Ferret-UI exhibits outstanding comprehension of UI screens and the capability to execute open-ended instructions. For model evaluation, we establish a comprehensive benchmark encompassing all the aforementioned tasks. Ferret-UI excels not only beyond most open-source UI MLLMs, but also surpasses GPT-4V on all the elementary UI tasks. △ Less

Submitted 8 April, 2024; originally announced April 2024.

Keyframer: Empowering Animation Design using Large Language Models

Authors: Tiffany Tseng, Ruijia Cheng, Jeffrey Nichols

Abstract: Large language models (LLMs) have the potential to impact a wide range of creative domains, but the application of LLMs to animation is underexplored and presents novel challenges such as how users might effectively describe motion in natural language. In this paper, we present Keyframer, a design tool for animating static images (SVGs) with natural language. Informed by interviews with profession… ▽ More Large language models (LLMs) have the potential to impact a wide range of creative domains, but the application of LLMs to animation is underexplored and presents novel challenges such as how users might effectively describe motion in natural language. In this paper, we present Keyframer, a design tool for animating static images (SVGs) with natural language. Informed by interviews with professional animation designers and engineers, Keyframer supports exploration and refinement of animations through the combination of prompting and direct editing of generated output. The system also enables users to request design variants, supporting comparison and ideation. Through a user study with 13 participants, we contribute a characterization of user prompting strategies, including a taxonomy of semantic prompt types for describing motion and a 'decomposed' prompting style where users continually adapt their goals in response to generated output.We share how direct editing along with prompting enables iteration beyond one-shot prompting interfaces common in generative tools today. Through this work, we propose how LLMs might empower a range of audiences to engage with animation creation. △ Less

Submitted 8 February, 2024; originally announced February 2024.

ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations

Authors: Yue Jiang, Eldon Schoop, Amanda Swearngin, Jeffrey Nichols

Abstract: Multimodal Vision-Language Models (VLMs) enable powerful applications from their fused understanding of images and language, but many perform poorly on UI tasks due to the lack of UI training data. In this paper, we adapt a recipe for generating paired text-image training data for VLMs to the UI domain by combining existing pixel-based methods with a Large Language Model (LLM). Unlike prior art, o… ▽ More Multimodal Vision-Language Models (VLMs) enable powerful applications from their fused understanding of images and language, but many perform poorly on UI tasks due to the lack of UI training data. In this paper, we adapt a recipe for generating paired text-image training data for VLMs to the UI domain by combining existing pixel-based methods with a Large Language Model (LLM). Unlike prior art, our method requires no human-provided annotations, and it can be applied to any dataset of UI screenshots. We generate a dataset of 335K conversational examples paired with UIs that cover Q&A, UI descriptions, and planning, and use it to fine-tune a conversational VLM for UI tasks. To assess the performance of our model, we benchmark it on UI element detection tasks, evaluate response quality, and showcase its applicability to multi-step UI navigation and planning. △ Less

Submitted 7 October, 2023; originally announced October 2023.

AXNav: Replaying Accessibility Tests from Natural Language

Authors: Maryam Taeb, Amanda Swearngin, Eldon Schoop, Ruijia Cheng, Yue Jiang, Jeffrey Nichols

Abstract: Developers and quality assurance testers often rely on manual testing to test accessibility features throughout the product lifecycle. Unfortunately, manual testing can be tedious, often has an overwhelming scope, and can be difficult to schedule amongst other development milestones. Recently, Large Language Models (LLMs) have been used for a variety of tasks including automation of UIs, however t… ▽ More Developers and quality assurance testers often rely on manual testing to test accessibility features throughout the product lifecycle. Unfortunately, manual testing can be tedious, often has an overwhelming scope, and can be difficult to schedule amongst other development milestones. Recently, Large Language Models (LLMs) have been used for a variety of tasks including automation of UIs, however to our knowledge no one has yet explored their use in controlling assistive technologies for the purposes of supporting accessibility testing. In this paper, we explore the requirements of a natural language based accessibility testing workflow, starting with a formative study. From this we build a system that takes as input a manual accessibility test (e.g., ``Search for a show in VoiceOver'') and uses an LLM combined with pixel-based UI Understanding models to execute the test and produce a chaptered, navigable video. In each video, to help QA testers we apply heuristics to detect and flag accessibility issues (e.g., Text size not increasing with Large Text enabled, VoiceOver navigation loops). We evaluate this system through a 10 participant user study with accessibility QA professionals who indicated that the tool would be very useful in their current work and performed tests similarly to how they would manually test the features. The study also reveals insights for future work on using LLMs for accessibility testing. △ Less

Submitted 4 March, 2024; v1 submitted 3 October, 2023; originally announced October 2023.

Comments: Accepted into Conference on Human Factors in Computing Systems (CHI) 2024, 22 pages, 7 figures

ACM Class: I.2

Towards Automated Accessibility Report Generation for Mobile Apps

Authors: Amanda Swearngin, Jason Wu, Xiaoyi Zhang, Esteban Gomez, Jen Coughenour, Rachel Stukenborg, Bhavya Garg, Greg Hughes, Adriana Hilliard, Jeffrey P. Bigham, Jeffrey Nichols

Abstract: Many apps have basic accessibility issues, like missing labels or low contrast. Automated tools can help app developers catch basic issues, but can be laborious or require writing dedicated tests. We propose a system, motivated by a collaborative process with accessibility stakeholders at a large technology company, to generate whole app accessibility reports by combining varied data collection me… ▽ More Many apps have basic accessibility issues, like missing labels or low contrast. Automated tools can help app developers catch basic issues, but can be laborious or require writing dedicated tests. We propose a system, motivated by a collaborative process with accessibility stakeholders at a large technology company, to generate whole app accessibility reports by combining varied data collection methods (e.g., app crawling, manual recording) with an existing accessibility scanner. Many such scanners are based on single-screen scanning, and a key problem in whole app accessibility reporting is to effectively de-duplicate and summarize issues collected across an app. To this end, we developed a screen grou** model with 96.9% accuracy (88.8% F1-score) and UI element matching heuristics with 97% accuracy (98.2% F1-score). We combine these technologies in a system to report and summarize unique issues across an app, and enable a unique pixel-based ignore feature to help engineers and testers better manage reported issues across their app's lifetime. We conducted a qualitative evaluation with 18 accessibility-focused engineers and testers which showed this system can enhance their existing accessibility testing toolkit and address key limitations in current accessibility scanning tools. △ Less

Submitted 16 October, 2023; v1 submitted 29 September, 2023; originally announced October 2023.

Comments: 24 pages, 8 figures

AI ATAC 1: An Evaluation of Prominent Commercial Malware Detectors

Authors: Robert A. Bridges, Brian Weber, Justin M. Beaver, Jared M. Smith, Miki E. Verma, Savannah Norem, Kevin Spakes, Cory Watson, Jeff A. Nichols, Brian Jewell, Michael. D. Iannacone, Chelsey Dunivan Stahl, Kelly M. T. Huffer, T. Sean Oesch

Abstract: This work presents an evaluation of six prominent commercial endpoint malware detectors, a network malware detector, and a file-conviction algorithm from a cyber technology vendor. The evaluation was administered as the first of the Artificial Intelligence Applications to Autonomous Cybersecurity (AI ATAC) prize challenges, funded by / completed in service of the US Navy. The experiment employed 1… ▽ More This work presents an evaluation of six prominent commercial endpoint malware detectors, a network malware detector, and a file-conviction algorithm from a cyber technology vendor. The evaluation was administered as the first of the Artificial Intelligence Applications to Autonomous Cybersecurity (AI ATAC) prize challenges, funded by / completed in service of the US Navy. The experiment employed 100K files (50/50% benign/malicious) with a stratified distribution of file types, including ~1K zero-day program executables (increasing experiment size two orders of magnitude over previous work). We present an evaluation process of delivering a file to a fresh virtual machine donning the detection technology, waiting 90s to allow static detection, then executing the file and waiting another period for dynamic detection; this allows greater fidelity in the observational data than previous experiments, in particular, resource and time-to-detection statistics. To execute all 800K trials (100K files $\times$ 8 tools), a software framework is designed to choreographed the experiment into a completely automated, time-synced, and reproducible workflow with substantial parallelization. A cost-benefit model was configured to integrate the tools' recall, precision, time to detection, and resource requirements into a single comparable quantity by simulating costs of use. This provides a ranking methodology for cyber competitions and a lens through which to reason about the varied statistical viewpoints of the results. These statistical and cost-model results provide insights on state of commercial malware detection. △ Less

Submitted 28 August, 2023; originally announced August 2023.

Data-driven Identification of Parametric Governing Equations of Dynamical Systems Using the Signed Cumulative Distribution Transform

Authors: Abu Hasnat Mohammad Rubaiyat, Duy H. Thai, Jonathan M. Nichols, Meredith N. Hutchinson, Samuel P. Wallen, Christina J. Naify, Nathan Geib, Michael R. Haberman, Gustavo K. Rohde

Abstract: This paper presents a novel data-driven approach to identify partial differential equation (PDE) parameters of a dynamical system. Specifically, we adopt a mathematical "transport" model for the solution of the dynamical system at specific spatial locations that allows us to accurately estimate the model parameters, including those associated with structural damage. This is accomplished by means o… ▽ More This paper presents a novel data-driven approach to identify partial differential equation (PDE) parameters of a dynamical system. Specifically, we adopt a mathematical "transport" model for the solution of the dynamical system at specific spatial locations that allows us to accurately estimate the model parameters, including those associated with structural damage. This is accomplished by means of a newly-developed mathematical transform, the signed cumulative distribution transform (SCDT), which is shown to convert the general nonlinear parameter estimation problem into a simple linear regression. This approach has the additional practical advantage of requiring no a priori knowledge of the source of the excitation (or, alternatively, the initial conditions). By using training data, we devise a coarse regression procedure to recover different PDE parameters from the PDE solution measured at a single location. Numerical experiments show that the proposed regression procedure is capable of detecting and estimating PDE parameters with superior accuracy compared to a number of recently developed machine learning methods. Furthermore, a damage identification experiment conducted on a publicly available dataset provides strong evidence of the proposed method's effectiveness in structural health monitoring (SHM) applications. The Python implementation of the proposed system identification technique is integrated as a part of the software package PyTransKit (https://github.com/rohdelab/PyTransKit). △ Less

Submitted 14 November, 2023; v1 submitted 23 August, 2023; originally announced August 2023.

Never-ending Learning of User Interfaces

Authors: Jason Wu, Rebecca Krosnick, Eldon Schoop, Amanda Swearngin, Jeffrey P. Bigham, Jeffrey Nichols

Abstract: Machine learning models have been trained to predict semantic information about user interfaces (UIs) to make apps more accessible, easier to test, and to automate. Currently, most models rely on datasets that are collected and labeled by human crowd-workers, a process that is costly and surprisingly error-prone for certain tasks. For example, it is possible to guess if a UI element is "tappable"… ▽ More Machine learning models have been trained to predict semantic information about user interfaces (UIs) to make apps more accessible, easier to test, and to automate. Currently, most models rely on datasets that are collected and labeled by human crowd-workers, a process that is costly and surprisingly error-prone for certain tasks. For example, it is possible to guess if a UI element is "tappable" from a screenshot (i.e., based on visual signifiers) or from potentially unreliable metadata (e.g., a view hierarchy), but one way to know for certain is to programmatically tap the UI element and observe the effects. We built the Never-ending UI Learner, an app crawler that automatically installs real apps from a mobile app store and crawls them to discover new and challenging training examples to learn from. The Never-ending UI Learner has crawled for more than 5,000 device-hours, performing over half a million actions on 6,000 apps to train three computer vision models for i) tappability prediction, ii) draggability prediction, and iii) screen similarity. △ Less

Submitted 16 August, 2023; originally announced August 2023.

The Mass-Temperature Relation for B and Early A Stars Based on IUE Spectra of Detached Eclipsing Binaries

Authors: Nancy Remage Evans, Mckenzie G. Ferrari, Joanna Kuraszkiewicz, Steven Silverberg, Joy Nichols, Guillermo Torres, Makenzi Fischbach

Abstract: Ultraviolet spectra were taken of 25 Detached Eclipsing Binaries (DEBs) with spectral types O, B, and early A with the International Ultraviolet Explorer (IUE) satellite in the 1150 to 1900 $Å$ region. The spectra were compared with BOSZ model atmospheres (Bohlin, et al. 2017). The composite spectra of the DEBs were modeled by a combination of models representing the hot and cool components, and t… ▽ More Ultraviolet spectra were taken of 25 Detached Eclipsing Binaries (DEBs) with spectral types O, B, and early A with the International Ultraviolet Explorer (IUE) satellite in the 1150 to 1900 $Å$ region. The spectra were compared with BOSZ model atmospheres (Bohlin, et al. 2017). The composite spectra of the DEBs were modeled by a combination of models representing the hot and cool components, and the temperatures of the hottest components of the systems were determined. From these temperatures a direct Mass-Temperature relation was obtained for stars close to the main sequence with solar metallicity for B and early A stars: log M/Msun = -5.90 $\pm$ 0.27 + (1.56 $\pm$ 0.07) x log T This relation allows a mass to be inferred for comparable stars from an ultraviolet spectrum. The five chemically peculiar Am stars in the sample have larger radii than normal A stars of the same mass. △ Less

Submitted 2 August, 2023; originally announced August 2023.

Comments: accepted by AJ

Report number: 04

Survey of X-rays from Massive Stars Observed at High Spectral Resolution with Chandra

Authors: Pragati Pradhan, David P. Huenemoerder, Richard Ignace, Joy S. Nichols, A. M. T. Pollock

Abstract: Identifying trends between observational data and the range of physical parameters of massive stars is a critical step to the still-elusive full understanding of the source, structure, and evolution of X-ray emission from the stellar winds, requiring a substantial sample size and systematic analysis methods. The \emph{Chandra} data archive as of 2022 contains 37 high resolution spectra of O, B, an… ▽ More Identifying trends between observational data and the range of physical parameters of massive stars is a critical step to the still-elusive full understanding of the source, structure, and evolution of X-ray emission from the stellar winds, requiring a substantial sample size and systematic analysis methods. The \emph{Chandra} data archive as of 2022 contains 37 high resolution spectra of O, B, and WR stars, observed with the \emph{Chandra}/HETGS and of sufficient quality to fit the continua and emission line profiles. Using a systematic approach to the data analysis, we explore morphological trends in the line profiles (i.e., O, Ne, Mg, Si) and find that the centroid offsets of resolved lines versus wavelength can be separated in three empirically-defined groups based on the amount of line broadening and centroid offset. Using \ion{Fe}{17} (15.01 Å, 17.05 Å) and \ion{Ne}{10} $α$ (12.13 Å) lines which are prevalent among the sample stars, we find a well-correlated linear trend of increasing Full Width Half Maximum (FWHM) with faster wind terminal velocity. The H-like/He-like total line flux ratio for strong lines displays different trends with spectral class depending on ion species. Some of the sources in our sample have peculiar properties (e.g., magnetic and $γ$ Cas-analogue stars) and we find that these sources stand out as outliers from more regular trends. Finally, our spectral analysis is presented summarily in terms of X-ray spectral energy distributions in specific luminosity for each source, plus tables of line identifications and fluxes. △ Less

Submitted 1 August, 2023; originally announced August 2023.

Comments: 78 pages, 46 figures, 41 tables, Accepted for publication in ApJ

The RS Oph outburst of 2021 monitored in X-rays with NICER

Authors: Marina Orio, Keith Gendreau, Morgan Giese, Gerardo Juna M. Luna, Jozef Magdolen, Tod E. Strohmayer, Andy E. Zhang, Diego Altamirano, Andrej Dobrotka, Teruaki Enoto, Elizabeth C. Ferrara, Richard Ignace, Sebastian heinz, Craig Markwardt, Joy S. Nichols, Micahel L. Parker, Dheerajay R. Pasham, Songpeng Pei, Pragati Pradhan, Ron Remillard, James F. Steiner, Francesco Tombesi

Abstract: The 2021 outburst of the symbiotic recurrent nova RS Oph was monitored with the Neutron Star Interior Composition Explorer Mission (NICER) in the 0.2-12 keV range from day one after the optical maximum, until day 88, producing an unprecedented, detailed view of the outburst development. The X-ray flux preceding the supersoft X-ray phase peaked almost 5 days after optical maximum and originated onl… ▽ More The 2021 outburst of the symbiotic recurrent nova RS Oph was monitored with the Neutron Star Interior Composition Explorer Mission (NICER) in the 0.2-12 keV range from day one after the optical maximum, until day 88, producing an unprecedented, detailed view of the outburst development. The X-ray flux preceding the supersoft X-ray phase peaked almost 5 days after optical maximum and originated only in shocked ejecta for 21 to 25 days. The emission was thermal; in the first 5 days only a non-collisional-ionization equilibrium model fits the spectrum, and a transition to equilibrium occurred between days 6 and 12. The ratio of peak X-rays flux measured in the NICER range to that measured with Fermi in the 60 MeV-500 GeV range was about 0.1, and the ratio to the peak flux measured with H.E.S.S. in the 250 GeV-2.5 TeV range was about 100. The central supersoft X-ray source (SSS), namely the shell hydrogen burning white dwarf (WD), became visible in the fourth week, initially with short flares. A huge increase in flux occurred on day 41, but the SSS flux remained variable. A quasi-periodic oscillation every ~35 s was always observed during the SSS phase, with variations in amplitude and a period drift that appeared to decrease in the end. The SSS has characteristics of a WD of mass >1 M(solar). Thermonuclear burning switched off shortly after day 75, earlier than in 2006 outburst. We discuss implications for the nova physics. △ Less

Submitted 21 July, 2023; originally announced July 2023.

Comments: Accepted for publication in the Astrophysical Journal

Three-dimensional receptivity of hypersonic sharp and blunt cones to free-stream planar waves using hierarchical input-output analysis

Authors: David A. Cook, Joseph W. Nichols

Abstract: Understanding the receptivity of hypersonic flows to free-stream disturbances is crucial for predicting laminar to turbulent boundary layer transition. Input-output analysis as a receptivity tool considers which free-stream disturbances lead to the largest response from the boundary layer using the global linear dynamics. Two technical challenges are addressed. First, we extend recent work by Kama… ▽ More Understanding the receptivity of hypersonic flows to free-stream disturbances is crucial for predicting laminar to turbulent boundary layer transition. Input-output analysis as a receptivity tool considers which free-stream disturbances lead to the largest response from the boundary layer using the global linear dynamics. Two technical challenges are addressed. First, we extend recent work by Kamal et al. (Kamal, 2023) and restrict the allowable forcing to physically realizable inputs via a free-stream boundary modification to the classic input-output formulation. Second, we develop a hierarchical input-output (H-IO) analysis which allows us to solve the three-dimensional problem at a fraction of the computational cost otherwise associated with directly inverting the fully three-dimensional resolvent operator. Next, we consider Mach 5.8 flows over a sharp cone and two blunt cones with 3.6 mm and 7.2 mm spherically blunt tips. H-IO correctly predicts that the sharp cone boundary layer is most receptive to slow acoustic waves at an optimal incidence angle of 10 degrees, validating the method. We then investigate the effect of free-stream disturbances on the blunt cone boundary layer, and identify two distinct vorticity-dominated receptivity mechanisms for the oblique first mode instability at 10 kHz and an entropy layer instability at 40 and 70 kHz. Our results reveal these receptivity processes to be highly three-dimensional in nature, involving both the nose-tip and excitation along narrow bands at certain azimuthal angles along the oblique shock downstream. We interpret these processes in terms of critical angles from linear shock/perturbation interaction theory. Finally, we show how these novel receptivity processes vary with frequency and nose tip bluntness, and demonstrate how this methodology might be applied to transition prediction from first principles. △ Less

Submitted 4 December, 2023; v1 submitted 5 June, 2023; originally announced June 2023.

Comments: Submitted to Physical Review Fluids

Linear Analysis of Boundary-Layer Instabilities on a Finned-Cone at Mach 6

Authors: Daniel B. Araya, Neal P. Bitter, Bradley M. Wheaton, Omar Kamal, Tim Colonius, Anthony Knutson, Heath Johnson, Joseph Nichols, Graham V. Candler, Vincenzo Russo, Christoph Brehm

Abstract: Boundary-layer instabilities for a finned cone at Mach=6, $Re=8.4 \times 10^6$ [m$^{-1}$], and zero incidence angle are examined using linear stability methods of varying fidelity and maturity, following earlier analysis presented in [doi.org/10.2514/6.2022-3247]. The geometry and laminar flow conditions correspond to experiments conducted at the Boeing Air Force Mach 6 Quiet Tunnel (BAM6QT) at Pu… ▽ More Boundary-layer instabilities for a finned cone at Mach=6, $Re=8.4 \times 10^6$ [m$^{-1}$], and zero incidence angle are examined using linear stability methods of varying fidelity and maturity, following earlier analysis presented in [doi.org/10.2514/6.2022-3247]. The geometry and laminar flow conditions correspond to experiments conducted at the Boeing Air Force Mach 6 Quiet Tunnel (BAM6QT) at Purdue University. Where possible, a common mean flow is utilized among the stability computations, and comparisons are made along the acreage of the cone where transition is first observed in the experiment. Stability results utilizing Linear Stability Theory (LST), planar Parabolized Stability Equations (planar-PSE), One-Way Navier Stokes (OWNS), forced direct numerical simulation (DNS), and Adaptive Mesh Refinement Wavepacket Tracking (AMR-WPT) are presented. A dominant three-dimensional vortex instability occurring at $\approx$ 250 kHz is identified that correlates well with experimental measurements of transition onset. With the exception of LST, all of the higher-fidelity linear methods considered in this work were consistent in predicting the initial growth and general structure of the vortex instability as it evolved downstream. Some of the challenges, opportunities, and development needs of the stability methods considered are discussed. △ Less

Submitted 19 March, 2023; originally announced March 2023.

Topological Deep Learning: A Review of an Emerging Paradigm

Authors: Ali Zia, Abdelwahed Khamis, James Nichols, Zeeshan Hayder, Vivien Rolland, Lars Petersson

Abstract: Topological data analysis (TDA) provides insight into data shape. The summaries obtained by these methods are principled global descriptions of multi-dimensional data whilst exhibiting stable properties such as robustness to deformation and noise. Such properties are desirable in deep learning pipelines but they are typically obtained using non-TDA strategies. This is partly caused by the difficul… ▽ More Topological data analysis (TDA) provides insight into data shape. The summaries obtained by these methods are principled global descriptions of multi-dimensional data whilst exhibiting stable properties such as robustness to deformation and noise. Such properties are desirable in deep learning pipelines but they are typically obtained using non-TDA strategies. This is partly caused by the difficulty of combining TDA constructs (e.g. barcode and persistence diagrams) with current deep learning algorithms. Fortunately, we are now witnessing a growth of deep learning applications embracing topologically-guided components. In this survey, we review the nascent field of topological deep learning by first revisiting the core concepts of TDA. We then explore how the use of TDA techniques has evolved over time to support deep learning frameworks, and how they can be integrated into different aspects of deep learning. Furthermore, we touch on TDA usage for analyzing existing deep models; deep topological analytics. Finally, we discuss the challenges and future prospects of topological deep learning. △ Less

Submitted 7 February, 2023; originally announced February 2023.

Comments: 7 pages and 2 references

WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics

Authors: Jason Wu, Siyan Wang, Siman Shen, Yi-Hao Peng, Jeffrey Nichols, Jeffrey P. Bigham

Abstract: Modeling user interfaces (UIs) from visual information allows systems to make inferences about the functionality and semantics needed to support use cases in accessibility, app automation, and testing. Current datasets for training machine learning models are limited in size due to the costly and time-consuming process of manually collecting and annotating UIs. We crawled the web to construct WebU… ▽ More Modeling user interfaces (UIs) from visual information allows systems to make inferences about the functionality and semantics needed to support use cases in accessibility, app automation, and testing. Current datasets for training machine learning models are limited in size due to the costly and time-consuming process of manually collecting and annotating UIs. We crawled the web to construct WebUI, a large dataset of 400,000 rendered web pages associated with automatically extracted metadata. We analyze the composition of WebUI and show that while automatically extracted data is noisy, most examples meet basic criteria for visual UI modeling. We applied several strategies for incorporating semantics found in web pages to increase the performance of visual UI understanding models in the mobile domain, where less labeled data is available: (i) element detection, (ii) screen classification and (iii) screen similarity. △ Less

Submitted 30 January, 2023; originally announced January 2023.

Comments: Accepted to CHI 2023. Dataset, code, and models release coming soon

Screen Correspondence: Map** Interchangeable Elements between UIs

Authors: Jason Wu, Amanda Swearngin, Xiaoyi Zhang, Jeffrey Nichols, Jeffrey P. Bigham

Abstract: Understanding user interface (UI) functionality is a useful yet challenging task for both machines and people. In this paper, we investigate a machine learning approach for screen correspondence, which allows reasoning about UIs by map** their elements onto previously encountered examples with known functionality and properties. We describe and implement a model that incorporates element semanti… ▽ More Understanding user interface (UI) functionality is a useful yet challenging task for both machines and people. In this paper, we investigate a machine learning approach for screen correspondence, which allows reasoning about UIs by map** their elements onto previously encountered examples with known functionality and properties. We describe and implement a model that incorporates element semantics, appearance, and text to support correspondence computation without requiring any labeled examples. Through a comprehensive performance evaluation, we show that our approach improves upon baselines by incorporating multi-modal properties of UIs. Finally, we show three example applications where screen correspondence facilitates better UI understanding for humans and machines: (i) instructional overlay generation, (ii) semantic UI element search, and (iii) automated interface testing. △ Less

Submitted 19 January, 2023; originally announced January 2023.

Transport Model for the Propagation of Partially-Coherent, Polarization-Gradient Vector Beams

Authors: J. M. Nichols, D. V. Nickel, G. K. Rohde, F. Bucholtz

Abstract: In a recent work [20], we predicted and experimentally validated a new physical mechanism for altering the propagation path of a monochromatic beam. Specifically, we showed that by properly tailoring the spatial distribution of the linear state of polarization transverse to the direction of propagation, the beam followed a curved trajectory in free space. Here we extend the model to the partially… ▽ More In a recent work [20], we predicted and experimentally validated a new physical mechanism for altering the propagation path of a monochromatic beam. Specifically, we showed that by properly tailoring the spatial distribution of the linear state of polarization transverse to the direction of propagation, the beam followed a curved trajectory in free space. Here we extend the model to the partially coherent, polychromatic case by redefining the beam amplitude, phase, and polarization angle as appropriate statistical quantities. In particular, we propose an entirely new definition of linear polarization gradient as an average over the third generalized Stokes parameter in the spatial frequency domain. In the new model, the beam curvature matches that of our previous work in the fully coherent case, but is predicted to gradually vanish as the beam loses coherence and becomes depolarized. The model also clearly predicts, however, that there does not exist a natural mechanism in free space or due to atmospheric turbulence that will cause significant depolarization. Simulated beam trajectories are shown for varying levels of initial partial coherence and for different polarization profiles. A new class of non-diffracting beams is also suggested by way of example. Lastly, as a byproduct of the derivation we generalize a previously known result concerning the free-space propagation of generalized Stokes parameters [13] by showing the result also holds for propagation in inhomogeneous media. △ Less

Submitted 10 January, 2023; originally announced January 2023.

Comments: 25 pages, 3 figures, 3 appendices

MSC Class: 35Q60; 78A60

Shocks in the outflow of the RS Oph 2021 eruption observed with X-ray gratings

Authors: Marina Orio, Ehud Behar, Juan Luna, Jeremy Drake, Jay Gallagher, Joy S. Nichols, Jan-Uwe Ness, Andrej Dobrotka, Joanna Mikolajewska, Massimo Della Valle, Rico Ignace, Roy Rahin

Abstract: The 2021 outburst of the symbiotic recurrent nova RS Oph was observed with the Chandra High Energy Transmission Gratings (HETG) on day 18 after optical maximum and with XMM-Newton and its Reflection Grating Spectrographs (RGS) on day 21, before the supersoft X-ray source emerged and when the emission was due to shocked ejecta. The absorbed flux in the HETG 1.3-31 Angstrom range was 2.6 x 10(-10) e… ▽ More The 2021 outburst of the symbiotic recurrent nova RS Oph was observed with the Chandra High Energy Transmission Gratings (HETG) on day 18 after optical maximum and with XMM-Newton and its Reflection Grating Spectrographs (RGS) on day 21, before the supersoft X-ray source emerged and when the emission was due to shocked ejecta. The absorbed flux in the HETG 1.3-31 Angstrom range was 2.6 x 10(-10) erg/cm(-2)/s, three orders of magnitude lower than the gamma-ray flux measured on the same date. The spectra are well fitted with two components of thermal plasma in collisional ionization equilibrium, one at a temperature ~0.75 keV, and the other at temperature in the 2.5-3.4 keV range. With the RGS we measured an average flux 1.53 x 10(-10) erg/cm(-2)/s in the 5-35 Angstrom range, but the flux in the continuum and especially in the lines in the 23-35 Angstrom range decreased during the 50 ks RGS exposure by almost 10%, indicating short term variability on hours' time scale. The RGS spectrum can be fitted with three thermal components, respectively at plasma temperature between 70 and 150 eV, 0.64 keV and 2.4 keV. The post-maximum epochs of the exposures fall between those of two grating spectra observed in the 2006 eruption on days 14 and 26: they are consistent with a similar spectral evolution, but in 2021 cooling seems to have been more rapid. Iron is depleted in the ejecta with respect to solar values, while nitrogen is enhanced. △ Less

Submitted 5 September, 2022; originally announced September 2022.

Comments: In press in the Astrophysical Journal

Testing SOAR Tools in Use

Authors: Robert A. Bridges, Ashley E. Rice, Sean Oesch, Jeff A. Nichols, Cory Watson, Kevin Spakes, Savannah Norem, Mike Huettel, Brian Jewell, Brian Weber, Connor Gannon, Olivia Bizovi, Samuel C Hollifield, Samantha Erwin

Abstract: Modern security operation centers (SOCs) rely on operators and a tapestry of logging and alerting tools with large scale collection and query abilities. SOC investigations are tedious as they rely on manual efforts to query diverse data sources, overlay related logs, and correlate the data into information and then document results in a ticketing system. Security orchestration, automation, and res… ▽ More Modern security operation centers (SOCs) rely on operators and a tapestry of logging and alerting tools with large scale collection and query abilities. SOC investigations are tedious as they rely on manual efforts to query diverse data sources, overlay related logs, and correlate the data into information and then document results in a ticketing system. Security orchestration, automation, and response (SOAR) tools are a new technology that promise to collect, filter, and display needed data; automate common tasks that require SOC analysts' time; facilitate SOC collaboration; and, improve both efficiency and consistency of SOCs. SOAR tools have never been tested in practice to evaluate their effect and understand them in use. In this paper, we design and administer the first hands-on user study of SOAR tools, involving 24 participants and 6 commercial SOAR tools. Our contributions include the experimental design, itemizing six characteristics of SOAR tools and a methodology for testing them. We describe configuration of the test environment in a cyber range, including network, user, and threat emulation; a full SOC tool suite; and creation of artifacts allowing multiple representative investigation scenarios to permit testing. We present the first research results on SOAR tools. We found that SOAR configuration is critical, as it involves creative design for data display and automation. We found that SOAR tools increased efficiency and reduced context switching during investigations, although ticket accuracy and completeness (indicating investigation quality) decreased with SOAR use. Our findings indicated that user preferences are slightly negatively correlated with their performance with the tool; overautomation was a concern of senior analysts, and SOAR tools that balanced automation with assisting a user to make decisions were preferred. △ Less

Submitted 14 February, 2023; v1 submitted 11 August, 2022; originally announced August 2022.

Journal ref: Computers & Security 2023

Reflow: Automatically Improving Touch Interactions in Mobile Applications through Pixel-based Refinements

Authors: Jason Wu, Titus Barik, Xiaoyi Zhang, Colin Lea, Jeffrey Nichols, Jeffrey P. Bigham

Abstract: Touch is the primary way that users interact with smartphones. However, building mobile user interfaces where touch interactions work well for all users is a difficult problem, because users have different abilities and preferences. We propose a system, Reflow, which automatically applies small, personalized UI adaptations, called refinements -- to mobile app screens to improve touch efficiency. R… ▽ More Touch is the primary way that users interact with smartphones. However, building mobile user interfaces where touch interactions work well for all users is a difficult problem, because users have different abilities and preferences. We propose a system, Reflow, which automatically applies small, personalized UI adaptations, called refinements -- to mobile app screens to improve touch efficiency. Reflow uses a pixel-based strategy to work with existing applications, and improves touch efficiency while minimally disrupting the design intent of the original application. Our system optimizes a UI by (i) extracting its layout from its screenshot, (ii) refining its layout, and (iii) re-rendering the UI to reflect these modifications. We conducted a user study with 10 participants and a heuristic evaluation with 6 experts and found that applications optimized by Reflow led to, on average, 9% faster selection time with minimal layout disruption. The results demonstrate that Reflow's refinements useful UI adaptations to improve touch interactions. △ Less

Submitted 15 July, 2022; originally announced July 2022.

Extracting Replayable Interactions from Videos of Mobile App Usage

Authors: Jieshan Chen, Amanda Swearngin, Jason Wu, Titus Barik, Jeffrey Nichols, Xiaoyi Zhang

Abstract: Screen recordings of mobile apps are a popular and readily available way for users to share how they interact with apps, such as in online tutorial videos, user reviews, or as attachments in bug reports. Unfortunately, both people and systems can find it difficult to reproduce touch-driven interactions from video pixel data alone. In this paper, we introduce an approach to extract and replay user… ▽ More Screen recordings of mobile apps are a popular and readily available way for users to share how they interact with apps, such as in online tutorial videos, user reviews, or as attachments in bug reports. Unfortunately, both people and systems can find it difficult to reproduce touch-driven interactions from video pixel data alone. In this paper, we introduce an approach to extract and replay user interactions in videos of mobile apps, using only pixel information in video frames. To identify interactions, we apply heuristic-based image processing and convolutional deep learning to segment screen recordings, classify the interaction in each segment, and locate the interaction point. To replay interactions on another device, we match elements on app screens using UI element detection. We evaluate the feasibility of our pixel-based approach using two datasets: the Rico mobile app dataset and a new dataset of 64 apps with both iOS and Android versions. We find that our end-to-end approach can successfully replay a majority of interactions (iOS--84.1%, Android--78.4%) on different devices, which is a step towards supporting a variety of scenarios, including automatically annotating interactions in existing videos, automated UI testing, and creating interactive app tutorials. △ Less

Submitted 8 July, 2022; originally announced July 2022.

Assembling a Cyber Range to Evaluate Artificial Intelligence / Machine Learning (AI/ML) Security Tools

Authors: Jeffrey A. Nichols, Kevin D. Spakes, Cory L. Watson, Robert A. Bridges

Abstract: In this case study, we describe the design and assembly of a cyber security testbed at Oak Ridge National Laboratory in Oak Ridge, TN, USA. The range is designed to provide agile reconfigurations to facilitate a wide variety of experiments for evaluations of cyber security tools -- particularly those involving AI/ML. In particular, the testbed provides realistic test environments while permitting… ▽ More In this case study, we describe the design and assembly of a cyber security testbed at Oak Ridge National Laboratory in Oak Ridge, TN, USA. The range is designed to provide agile reconfigurations to facilitate a wide variety of experiments for evaluations of cyber security tools -- particularly those involving AI/ML. In particular, the testbed provides realistic test environments while permitting control and programmatic observations/data collection during the experiments. We have designed in the ability to repeat the evaluations, so additional tools can be evaluated and compared at a later time. The system is one that can be scaled up or down for experiment sizes. At the time of the conference we will have completed two full-scale, national, government challenges on this range. These challenges are evaluating the performance and operating costs for AI/ML-based cyber security tools for application into large, government-sized networks. These evaluations will be described as examples providing motivation and context for various design decisions and adaptations we have made. The first challenge measured end-point security tools against 100K file samples (benignware and malware) chosen across a range of file types. The second is an evaluation of network intrusion detection systems efficacy in identifying multi-step adversarial campaigns -- involving reconnaissance, penetration and exploitations, lateral movement, etc. -- with varying levels of covertness in a high-volume business network. The scale of each of these challenges requires automation systems to repeat, or simultaneously mirror identical the experiments for each ML tool under test. Providing an array of easy-to-difficult malicious activity for sussing out the true abilities of the AI/ML tools has been a particularly interesting and challenging aspect of designing and executing these challenge events. △ Less

Submitted 20 January, 2022; originally announced January 2022.

Comments: ICCWS 2021 16th International Conference on Cyber Warfare and Security. Academic Conferences Limited, 2021

Sketch-based Creativity Support Tools using Deep Learning

Authors: Forrest Huang, Eldon Schoop, David Ha, Jeffrey Nichols, John Canny

Abstract: Sketching is a natural and effective visual communication medium commonly used in creative processes. Recent developments in deep-learning models drastically improved machines' ability in understanding and generating visual content. An exciting area of development explores deep-learning approaches used to model human sketches, opening opportunities for creative applications. This chapter describes… ▽ More Sketching is a natural and effective visual communication medium commonly used in creative processes. Recent developments in deep-learning models drastically improved machines' ability in understanding and generating visual content. An exciting area of development explores deep-learning approaches used to model human sketches, opening opportunities for creative applications. This chapter describes three fundamental steps in develo** deep-learning-driven creativity support tools that consumes and generates sketches: 1) a data collection effort that generated a new paired dataset between sketches and mobile user interfaces; 2) a sketch-based user interface retrieval system adapted from state-of-the-art computer vision techniques; and, 3) a conversational sketching system that supports the novel interaction of a natural-language-based sketch/critique authoring process. In this chapter, we survey relevant prior work in both the deep-learning and human-computer-interaction communities, document the data collection process and the systems' architectures in detail, present qualitative and quantitative results, and paint the landscape of several future research directions in this exciting area. △ Less

Submitted 18 November, 2021; originally announced November 2021.

Comments: Preprint of chapter in published in "Artificial Intelligence for Human Computer Interaction: A Modern Approach". arXiv admin note: substantial text overlap with arXiv:2005.07781

Vector Beam Bending

Authors: J. M. Nichols, D. V. Nickel, F. Bucholtz

Abstract: It is well known that the trajectory of a light beam can curve if the medium in which the beam propagates is inhomogeneous or if the local spacetime is curved. In this paper we present a third possibility and show that a properly-prepared optical beam follows a curved trajectory even in free space where the spacetime is locally flat. By casting the problem in terms of transport of optical intensit… ▽ More It is well known that the trajectory of a light beam can curve if the medium in which the beam propagates is inhomogeneous or if the local spacetime is curved. In this paper we present a third possibility and show that a properly-prepared optical beam follows a curved trajectory even in free space where the spacetime is locally flat. By casting the problem in terms of transport of optical intensity, we show that a beam comprising a gradient in state of polarization to second order or higher (i.e., a specifically prepared vector beam) must also follow a curved trajectory. The effect is confirmed experimentally for the case of a second-order gradient in a linearly polarized vector beam. △ Less

Submitted 30 October, 2021; v1 submitted 27 October, 2021; originally announced October 2021.

Comments: 4 pages, 2 figures

MSC Class: 35Q60; 78A60

Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots

Authors: Jason Wu, Xiaoyi Zhang, Jeff Nichols, Jeffrey P. Bigham

Abstract: Automated understanding of user interfaces (UIs) from their pixels can improve accessibility, enable task automation, and facilitate interface design without relying on developers to comprehensively provide metadata. A first step is to infer what UI elements exist on a screen, but current approaches are limited in how they infer how those elements are semantically grouped into structured interface… ▽ More Automated understanding of user interfaces (UIs) from their pixels can improve accessibility, enable task automation, and facilitate interface design without relying on developers to comprehensively provide metadata. A first step is to infer what UI elements exist on a screen, but current approaches are limited in how they infer how those elements are semantically grouped into structured interface definitions. In this paper, we motivate the problem of screen parsing, the task of predicting UI elements and their relationships from a screenshot. We describe our implementation of screen parsing and provide an effective training procedure that optimizes its performance. In an evaluation comparing the accuracy of the generated output, we find that our implementation significantly outperforms current systems (up to 23%). Finally, we show three example applications that are facilitated by screen parsing: (i) UI similarity search, (ii) accessibility enhancement, and (iii) code generation from UI screenshots. △ Less

Submitted 17 September, 2021; originally announced September 2021.

Using in-situ solar-wind observations to generate inner-boundary conditions to outer-heliosphere simulations, 1: Dynamic time war** applied to synthetic observations

Authors: Mathew Owens, Jonathan Nichols

Abstract: The structure and dynamics of the magnetospheres of the outer planets, particularly Saturn and Jupiter, have been explored both through remote and in-situ observations. Interpreting these observations often necessitates simultaneous knowledge of the solar-wind conditions im**ing on the magnetosphere. Without an available upstream monitor, solar-wind context is typically provided using models ini… ▽ More The structure and dynamics of the magnetospheres of the outer planets, particularly Saturn and Jupiter, have been explored both through remote and in-situ observations. Interpreting these observations often necessitates simultaneous knowledge of the solar-wind conditions im**ing on the magnetosphere. Without an available upstream monitor, solar-wind context is typically provided using models initiated with either the output of magnetogram-constrained coronal models or, more commonly, in-situ observations from 1 AU. While 1-AU observations provide a direct measure of solar wind conditions, they are single-point observations and thus require interpolation to provide inputs to outer-heliosphere solar-wind models. In this study we test the different interpolation methods using synthetic 1-AU observations of time-evolving solar-wind structure. The simplest method is "corotation", which assumes solar-wind structure is steady state and rotates with the Sun. This method of reconstruction produces discontinuities in the solar-wind inputs as new observations become available. This can be reduced by corotating both back and forward in time, but this still introduces large errors in the magnitude and timing of solar wind streams. We show how the dynamic time war** (DTW) algorithm can provide around an order-of-magnitude improvement in solar-wind inputs to outer-heliosphere model from in-situ observations near 1 AU. This is intended to build the foundation for further work demonstrating and validating methods to improve inner-boundary conditions to outer-heliosphere solar-wind models, including dealing with solar wind transients and quantifying the improvements at Saturn and Jupiter. △ Less

Submitted 8 September, 2021; originally announced September 2021.

Comments: 9 pages, 11 figures

Searching for Saturn's X-rays during a rare Jupiter Magnetotail Crossing using Chandra

Authors: D. M. Weigt, W. R. Dunn, C. M. Jackman, R. Kraft, G. Branduardi-Raymont, J. D. Nichols, A. D. Wibisono, M. F. Vogt, G. R. Gladstone

Abstract: Every 19 years, Saturn passes through Jupiter's 'flap**' magnetotail. Here, we report Chandra X-ray observations of Saturn planned to coincide with this rare planetary alignment and to analyse Saturn's magnetospheric response when transitioning to this unique parameter space. We analyse three Director's Discretionary Time (DDT) observations from the High Resolution Camera (HRC-I) on-board Chandr… ▽ More Every 19 years, Saturn passes through Jupiter's 'flap**' magnetotail. Here, we report Chandra X-ray observations of Saturn planned to coincide with this rare planetary alignment and to analyse Saturn's magnetospheric response when transitioning to this unique parameter space. We analyse three Director's Discretionary Time (DDT) observations from the High Resolution Camera (HRC-I) on-board Chandra, taken on November 19, 21 and 23 2020 with the aim to find auroral and/or disk emissions. We infer the conditions in the kronian system by looking at coincident soft X-ray solar flux data from the Geostationary Operational Environmental Satellite (GOES) and Hubble Space Telescope (HST) observations of Saturn's ultraviolet (UV) auroral emissions. The large Saturn-Sun-Earth angle during this time would mean that most flares from the Earth-facing side of the Sun would not have impacted Saturn. We find no significant detection of Saturn's disk or auroral emissions in any of our observations. We calculate the 3$σ$ upper band energy flux of Saturn during this time to be 0.9 - 3.04 $\times$ 10$^{14}$ erg cm$^{-2}$ s$^{-1}$ which agrees with fluxes found from previous modelled spectra of the disk emissions. We conclude by discussing the implications of this non-detection and how it is imperative that the next fleet of X-ray telescope (such as Athena and the Lynx mission concept) continue to observe Saturn with their improved spatial and spectral resolution and very enhanced sensitivity to help us finally solve the mysteries behind Saturn's apparently elusive X-ray aurora. △ Less

Submitted 9 June, 2021; originally announced June 2021.

Comments: 8 pages, 3 figures, accepted for publication in MNRAS

The Colliding Winds of WR 25 in High Resolution X-rays

Authors: Pragati Pradhan, David P. Huenemoerder, Richard Ignace, A. M. T Pollock, Joy S. Nichols

Abstract: WR 25 is a colliding-wind binary star system comprised of a very massive O2.5If*/WN6 primary and an O-star secondary in a 208-day period eccentric orbit. These hot stars have strong, highly-supersonic winds which interact to form a bright X-ray source from wind-collision-shocks whose conditions change with stellar separation. Different views through the WR and O star winds are afforded with orbita… ▽ More WR 25 is a colliding-wind binary star system comprised of a very massive O2.5If*/WN6 primary and an O-star secondary in a 208-day period eccentric orbit. These hot stars have strong, highly-supersonic winds which interact to form a bright X-ray source from wind-collision-shocks whose conditions change with stellar separation. Different views through the WR and O star winds are afforded with orbital phase as the stars move about their orbits, allowing for exploration of wind structure in ways not easy or even possible for single stars. We have analyzed an on-axis Chandra/HETGS spectrum of WR 25 obtained shortly before periastron when the X-rays emanating from the system are the brightest. From the on-axis observations, we constrain the line fluxes, centroids, and widths of various emission lines, including He-triplets of Si XIII and Mg XI. We have also been able to include several serendipitous off-axis HETG spectra from the archive and study their flux variation with phase. This is the first report on high-resolution spectral studies of WR 25 in X-rays. △ Less

Submitted 8 March, 2021; originally announced March 2021.

Comments: 27 pages, 10 figures, submitted to ApJ, comments welcome

Coarse reduced model selection for nonlinear state estimation

Authors: James A. Nichols

Abstract: State estimation is the task of approximately reconstructing a solution $u$ of a parametric partial differential equation when the parameter vector $y$ is unknown and the only information is $m$ linear measurements of $u$. In [Cohen et. al., 2021] the authors proposed a method to use a family of linear reduced spaces as a generalised nonlinear reduced model for state estimation. A computable surro… ▽ More State estimation is the task of approximately reconstructing a solution $u$ of a parametric partial differential equation when the parameter vector $y$ is unknown and the only information is $m$ linear measurements of $u$. In [Cohen et. al., 2021] the authors proposed a method to use a family of linear reduced spaces as a generalised nonlinear reduced model for state estimation. A computable surrogate distance is used to evaluate which linear estimate lies closest to a true solution of the PDE problem. In this paper we propose a strategy of coarse computation of the surrogate distance while maintaining a fine mesh reduced model, as the computational cost of the surrogate distance is large relative to the reduced modelling task. We demonstrate numerically that the error induced by the coarse distance is dominated by other approximation errors. △ Less

Submitted 5 March, 2021; originally announced March 2021.

Comments: 10 pages, 5 figures. Cconference report

MSC Class: 65M32; 65M15; 68U20; 65D99

Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels

Authors: Xiaoyi Zhang, Lilian de Greef, Amanda Swearngin, Samuel White, Kyle Murray, Lisa Yu, Qi Shan, Jeffrey Nichols, Jason Wu, Chris Fleizach, Aaron Everitt, Jeffrey P. Bigham

Abstract: Many accessibility features available on mobile platforms require applications (apps) to provide complete and accurate metadata describing user interface (UI) components. Unfortunately, many apps do not provide sufficient metadata for accessibility features to work as expected. In this paper, we explore inferring accessibility metadata for mobile apps from their pixels, as the visual interfaces of… ▽ More Many accessibility features available on mobile platforms require applications (apps) to provide complete and accurate metadata describing user interface (UI) components. Unfortunately, many apps do not provide sufficient metadata for accessibility features to work as expected. In this paper, we explore inferring accessibility metadata for mobile apps from their pixels, as the visual interfaces often best reflect an app's full functionality. We trained a robust, fast, memory-efficient, on-device model to detect UI elements using a dataset of 77,637 screens (from 4,068 iPhone apps) that we collected and annotated. To further improve UI detections and add semantic information, we introduced heuristics (e.g., UI grou** and ordering) and additional models (e.g., recognize UI content, state, interactivity). We built Screen Recognition to generate accessibility metadata to augment iOS VoiceOver. In a study with 9 screen reader users, we validated that our approach improves the accessibility of existing mobile apps, enabling even previously inaccessible apps to be used. △ Less

Submitted 13 January, 2021; originally announced January 2021.

Beyond the Hype: A Real-World Evaluation of the Impact and Cost of Machine Learning-Based Malware Detection

Authors: Robert A. Bridges, Sean Oesch, Miki E. Verma, Michael D. Iannacone, Kelly M. T. Huffer, Brian Jewell, Jeff A. Nichols, Brian Weber, Justin M. Beaver, Jared M. Smith, Daniel Scofield, Craig Miles, Thomas Plummer, Mark Daniell, Anne M. Tall

Abstract: In this paper, we present a scientific evaluation of four prominent malware detection tools to assist an organization with two primary questions: To what extent do ML-based tools accurately classify previously- and never-before-seen files? Is it worth purchasing a network-level malware detector? To identify weaknesses, we tested each tool against 3,536 total files (2,554 or 72\% malicious, 982 or… ▽ More In this paper, we present a scientific evaluation of four prominent malware detection tools to assist an organization with two primary questions: To what extent do ML-based tools accurately classify previously- and never-before-seen files? Is it worth purchasing a network-level malware detector? To identify weaknesses, we tested each tool against 3,536 total files (2,554 or 72\% malicious, 982 or 28\% benign) of a variety of file types, including hundreds of malicious zero-days, polyglots, and APT-style files, delivered on multiple protocols. We present statistical results on detection time and accuracy, consider complementary analysis (using multiple tools together), and provide two novel applications of the recent cost-benefit evaluation procedure of Iannacone \& Bridges. While the ML-based tools are more effective at detecting zero-day files and executables, the signature-based tool may still be an overall better option. Both network-based tools provide substantial (simulated) savings when paired with either host tool, yet both show poor detection rates on protocols other than HTTP or SMTP. Our results show that all four tools have near-perfect precision but alarmingly low recall, especially on file types other than executables and office files -- 37% of malware tested, including all polyglot files, were undetected. Priorities for researchers and takeaways for end users are given. △ Less

Submitted 17 August, 2022; v1 submitted 16 December, 2020; originally announced December 2020.

Comments: Includes Actionable Takeaways for SOCs

Journal ref: Digital Threats: Research and Practice 2023

The search for radio emission from the exoplanetary systems 55 Cancri, $\upsilon$ Andromedae, and $τ$ Boötis using LOFAR beam-formed observations

Authors: Jake D. Turner, Philippe Zarka, Jean-Mathias Grießmeier, Joseph Lazio, Baptiste Cecconi, J. Emilio Enriquez, Julien N. Girard, Ray Jayawardhana, Laurent Lamy, Jonathan D. Nichols, Imke de Pater

Abstract: Observing planetary auroral radio emission is the most promising method to detect exoplanetary magnetic fields, the knowledge of which will provide valuable insights into the planet's interior structure, atmospheric escape, and habitability. We present LOFAR-LBA circularly polarized beamformed observations of the exoplanetary systems 55 Cancri, $\upsilon$ Andromedae, and $τ$ Boötis. We tentatively… ▽ More Observing planetary auroral radio emission is the most promising method to detect exoplanetary magnetic fields, the knowledge of which will provide valuable insights into the planet's interior structure, atmospheric escape, and habitability. We present LOFAR-LBA circularly polarized beamformed observations of the exoplanetary systems 55 Cancri, $\upsilon$ Andromedae, and $τ$ Boötis. We tentatively detect circularly polarized bursty emission from the $τ$ Boötis system in the range 14-21 MHz with a flux density of $\sim$890 mJy and with a significance of $\sim$3$σ$. For this detection, no signal is seen in the OFF-beams, and we do not find any potential causes which might cause false positives. We also tentatively detect slowly variable circularly polarized emission from $τ$ Boötis in the range 21-30 MHz with a flux density of $\sim$400 mJy and with a statistical significance of $>$8$σ$. The slow emission is structured in the time-frequency plane and shows an excess in the ON-beam with respect to the two simultaneous OFF-beams. Close examination casts some doubts on the reality of the slowly varying signal. We discuss in detail all the arguments for and against an actual detection. Furthermore, a $\sim$2$σ$ marginal signal is found from the $\upsilon$ Andromedae system and no signal is detected from the 55 Cancri system. Assuming the detected signals are real, we discuss their potential origin. Their source probably is the $τ$ Bootis planetary system, and a possible explanation is radio emission from the exoplanet $τ$ Bootis b via the cyclotron maser mechanism. Assuming a planetary origin, we derived limits for the planetary polar surface magnetic field strength, finding values compatible with theoretical predictions. Further low-frequency observations are required to confirm this possible first detection of an exoplanetary radio signal. [Abridged] △ Less

Submitted 14 December, 2020; originally announced December 2020.

Comments: Accepted for publication in A&A (Oct. 22, 2020). 29 pages (15 pgs main text), 17 figures, 8 tables, 10 appendices

Journal ref: A&A 645, A59 (2021)

Correlated X-ray and optical variability in the O-type supergiant zeta Puppis

Authors: Joy S. Nichols, Yael Naze, David P. Huenemoerder, Anthony F. J. Moffat, Nathan Miller, Jennifer Lauer, Richard Ignace, Ken Gayley, Tahina Ramiaramanantsoa, Lidia Oskinova, Wolf-Rainer Hamann, Noel D. Richardson, Wayne L. Waldron, Matthew Dahmer

Abstract: Analysis of the recent long exposure Chandra X-ray observation of the early-type O star zeta Pup shows clear variability with a period previously reported in optical photometric studies. These 813 ks of HETG observations taken over a roughly one year time span have two signals of periodic variability: a high significance period of 1.7820 +/- 0.0008 day, and a marginal detection of periodic behavio… ▽ More Analysis of the recent long exposure Chandra X-ray observation of the early-type O star zeta Pup shows clear variability with a period previously reported in optical photometric studies. These 813 ks of HETG observations taken over a roughly one year time span have two signals of periodic variability: a high significance period of 1.7820 +/- 0.0008 day, and a marginal detection of periodic behavior close to either 5 day or 6 day period. A BRITE-Constellation nanosatellite optical photometric monitoring, using near-contemporaneous observations to the Chandra data, confirms a 1.78060 +/- 0.00088 day period for this star. The optical period coincides with the new Chandra period within their error ranges, demonstrating a link between these two wavebands and providing a powerful lever for probing the photosphere/wind connection in this star. The phase lag of the X-ray maximum relative to the optical maximum is approximately phi=0.45, but consideration of secondary maxima in both datasets indicates possibly two hot spots on the star with an X-ray phase lag of phi=0.1 each. The details of this periodic variation of the X-rays are probed by displaying a phased and trailed X-ray spectrum and by constructing phased light curves for wavelength bands within the HETG spectral coverage, ranging down to bands encompassing groups of emission lines. We propose that the 1.78 day period is the stellar rotation period and explore how stellar bright spots and associated co-rotating interacting regions or CIRs could explain the modulation of the optical and X-ray output for this star and their phase difference. △ Less

Submitted 13 November, 2020; originally announced November 2020.

Comments: 17 pages; 10 figures

Nonlinear reduced models for state and parameter estimation

Authors: Albert Cohen, Wolfgang Dahmen, Olga Mula, James Nichols

Abstract: State estimation aims at approximately reconstructing the solution $u$ to a parametrized partial differential equation from $m$ linear measurements, when the parameter vector $y$ is unknown. Fast numerical recovery methods have been proposed based on reduced models which are linear spaces of moderate dimension $n$ which are tailored to approximate the solution manifold $\mathcal{M}$ where the solu… ▽ More State estimation aims at approximately reconstructing the solution $u$ to a parametrized partial differential equation from $m$ linear measurements, when the parameter vector $y$ is unknown. Fast numerical recovery methods have been proposed based on reduced models which are linear spaces of moderate dimension $n$ which are tailored to approximate the solution manifold $\mathcal{M}$ where the solution sits. These methods can be viewed as deterministic counterparts to Bayesian estimation approaches, and are proved to be optimal when the prior is expressed by approximability of the solution with respect to the reduced model. However, they are inherently limited by their linear nature, which bounds from below their best possible performance by the Kolmogorov width $d_m(\mathcal{M})$ of the solution manifold. In this paper we propose to break this barrier by using simple nonlinear reduced models that consist of a finite union of linear spaces $V_k$, each having dimension at most $m$ and leading to different estimators $u_k^*$. A model selection mechanism based on minimizing the PDE residual over the parameter space is used to select from this collection the final estimator $u^*$. Our analysis shows that $u^*$ meets optimal recovery benchmarks that are inherent to the solution manifold and not tied to its Kolmogorov width. The residual minimization procedure is computationally simple in the relevant case of affine parameter dependence in the PDE. In addition, it results in an estimator $y^*$ for the unknown parameter vector. In this setting, we also discuss an alternating minimization (coordinate descent) algorithm for joint state and parameter estimation, that potentially improves the quality of both estimators. △ Less

Submitted 24 November, 2020; v1 submitted 6 September, 2020; originally announced September 2020.

NWChem: Past, Present, and Future

Authors: E. Aprà, E. J. Bylaska, W. A. de Jong, N. Govind, K. Kowalski, T. P. Straatsma, M. Valiev, H. J. J. van Dam, Y. Alexeev, J. Anchell, V. Anisimov, F. W. Aquino, R. Atta-Fynn, J. Autschbach, N. P. Bauman, J. C. Becca, D. E. Bernholdt, K. Bhaskaran-Nair, S. Bogatko, P. Borowski, J. Boschen, J. Brabec, A. Bruner, E. Cauët, Y. Chen , et al. (89 additional authors not shown)

Abstract: Specialized computational chemistry packages have permanently reshaped the landscape of chemical and materials science by providing tools to support and guide experimental efforts and for the prediction of atomistic and electronic properties. In this regard, electronic structure packages have played a special role by using first-principledriven methodologies to model complex chemical and materials… ▽ More Specialized computational chemistry packages have permanently reshaped the landscape of chemical and materials science by providing tools to support and guide experimental efforts and for the prediction of atomistic and electronic properties. In this regard, electronic structure packages have played a special role by using first-principledriven methodologies to model complex chemical and materials processes. Over the last few decades, the rapid development of computing technologies and the tremendous increase in computational power have offered a unique chance to study complex transformations using sophisticated and predictive many-body techniques that describe correlated behavior of electrons in molecular and condensed phase systems at different levels of theory. In enabling these simulations, novel parallel algorithms have been able to take advantage of computational resources to address the polynomial scaling of electronic structure methods. In this paper, we briefly review the NWChem computational chemistry suite, including its history, design principles, parallel tools, current capabilities, outreach and outlook. △ Less

Submitted 26 May, 2020; v1 submitted 24 April, 2020; originally announced April 2020.

Comments: This article appeared in volume 152, issue 18, page 184102 of the Journal of Chemical Physics. It can be found at https://doi.org/10.1063/5.0004997

Journal ref: J. Chem. Phys., 152, 184102 (2020)

Auroral diagnosis of solar wind interaction with Jupiter's magnetosphere

Authors: Z. H. Yao, B. Bonfond, D. Grodent, E. Chané, W. R. Dunn, W. S. Kurth, J. E. P. Connerney, J. D. Nichols, B. Palmaerts, R. L. Guo, G. B. Hospodarsky, B. H. Mauk, T. Kimura, S. J. Bolton

Abstract: Although mass and energy in Jupiter's magnetosphere mostly come from the innermost Galilean moon Io's volcanic activities, solar wind perturbations can play crucial roles in releasing the magnetospheric energy and powering aurorae in Jupiter's polar regions. The systematic response of aurora to solar wind compression remains poorly understood. Here we report the analysis of a set of auroral images… ▽ More Although mass and energy in Jupiter's magnetosphere mostly come from the innermost Galilean moon Io's volcanic activities, solar wind perturbations can play crucial roles in releasing the magnetospheric energy and powering aurorae in Jupiter's polar regions. The systematic response of aurora to solar wind compression remains poorly understood. Here we report the analysis of a set of auroral images with contemporaneous in situ magnetopause detections. We distinguish two types of auroral enhancements: a transient localized one and a long-lasting global one. We show that only the latter systematically appears under a compressed magnetopause, while the localized auroral expansion could occur during an expanded magnetopause. Moreover, we directly examine previous theories on how solar wind compressions enhance auroral emissions. Our results demonstrate that auroral morphologies can be diagnostic of solar wind conditions at planets when in situ measurements are not possible. △ Less

Submitted 21 April, 2020; originally announced April 2020.

Radon cumulative distribution transform subspace modeling for image classification

Authors: Mohammad Shifat-E-Rabbi, Xuwang Yin, Abu Hasnat Mohammad Rubaiyat, Shiying Li, Soheil Kolouri, Akram Aldroubi, Jonathan M. Nichols, Gustavo K. Rohde

Abstract: We present a new supervised image classification method applicable to a broad class of image deformation models. The method makes use of the previously described Radon Cumulative Distribution Transform (R-CDT) for image data, whose mathematical properties are exploited to express the image data in a form that is more suitable for machine learning. While certain operations such as translation, scal… ▽ More We present a new supervised image classification method applicable to a broad class of image deformation models. The method makes use of the previously described Radon Cumulative Distribution Transform (R-CDT) for image data, whose mathematical properties are exploited to express the image data in a form that is more suitable for machine learning. While certain operations such as translation, scaling, and higher-order transformations are challenging to model in native image space, we show the R-CDT can capture some of these variations and thus render the associated image classification problems easier to solve. The method -- utilizing a nearest-subspace algorithm in R-CDT space -- is simple to implement, non-iterative, has no hyper-parameters to tune, is computationally efficient, label efficient, and provides competitive accuracies to state-of-the-art neural networks for many types of classification problems. In addition to the test accuracy performances, we show improvements (with respect to neural network-based methods) in terms of computational efficiency (it can be implemented without the use of GPUs), number of training samples needed for training, as well as out-of-distribution generalization. The Python code for reproducing our results is available at https://github.com/rohdelab/rcdt_ns_classifier. △ Less

Submitted 2 March, 2022; v1 submitted 7 April, 2020; originally announced April 2020.

Comments: 14 pages, 11 figures

Magnetohydrodynamic modelling of star-planet interaction and associated auroral radio emission

Authors: Sam Turnpenney, Jonathan D. Nichols, Graham A. Wynn, Xianzhe Jia

Abstract: We present calculations of auroral radio powers of magnetised hot Jupiters orbiting Sun-like stars, computed using global magnetohydrodynamic (MHD) modelling of the magnetospheric and ionospheric convection arising from the interaction between the magnetosphere and the stellar wind. Exoplanetary auroral radio powers are traditionally estimated using empirical or analytically-derived relations, suc… ▽ More We present calculations of auroral radio powers of magnetised hot Jupiters orbiting Sun-like stars, computed using global magnetohydrodynamic (MHD) modelling of the magnetospheric and ionospheric convection arising from the interaction between the magnetosphere and the stellar wind. Exoplanetary auroral radio powers are traditionally estimated using empirical or analytically-derived relations, such as the Radiometric Bode's Law (RBL), which relates radio power to the magnetic or kinetic energy dissipated in the stellar wind-planet interaction. Such methods risk an oversimplification of the magnetospheric electrodynamics giving rise to radio emission. As the next step toward a self-consistent picture, we model the stellar wind-magnetosphere-ionosphere coupling currents using a 3D MHD model. We compute electron-cyclotron maser instability-driven emission from the calculated ionospheric field-aligned current density. We show that the auroral radio power is highly sensitive to interplanetary magnetic field (IMF) strength, and that the emission is saturated for plausible hot Jupiter Pedersen conductances, indicating that radio power may be largely independent of ionospheric conductance. We estimate peak radio powers of $10^{14}$ W from a planet exposed to an IMF strength of $10^3$ nT, implying flux densities at a distance of 15 pc from Earth potentially detectable with current and future radio telescopes. We also find a relation between radio power and planetary orbital distance that is broadly consistent with results from previous analytic models of magnetosphere-ionosphere coupling at hot Jupiters, and indicates that the RBL likely overestimates the radio powers by up to two orders of magnitude in the hot Jupiter regime △ Less

Submitted 22 March, 2020; originally announced March 2020.

Comments: 13 pages, 10 figures

A Deep Exposure in High Resolution X-Rays Reveals the Hottest Plasma in the $ζ\,$Puppis Wind

Authors: David P. Huenemoerder, Richard Ignace, Nathan A. Miller, Kenneth G. Gayley, Wolf-Rainer Hamann, Jennifer Lauer, Anthony F. J. Moffat, Yaël Nazé, Joy S. Nichols, Lidia Oskinova, Noel D. Richardson, Wayne Waldron

Abstract: We have obtained a very deep exposure (813 ks) of $ζ\,$Puppis (O4 supergiant) with the Chandra/HETG Spectrometer. Here we report on analysis of the 1-9 Å region, especially well suited for Chandra, which has a significant contribution from continuum emission between well separated emission lines from high-ionization species. These data allow us to study the hottest plasma present through the conti… ▽ More We have obtained a very deep exposure (813 ks) of $ζ\,$Puppis (O4 supergiant) with the Chandra/HETG Spectrometer. Here we report on analysis of the 1-9 Å region, especially well suited for Chandra, which has a significant contribution from continuum emission between well separated emission lines from high-ionization species. These data allow us to study the hottest plasma present through the continuum shape and emission line strengths. Assuming a powerlaw emission measure distribution which has a high-temperature cut-off, we find that the emission is consistent with a thermal spectrum having a maximum temperature of 12 MK. This implies an effective wind shock velocity of $900\,\mathrm{km\,s^{-1}}$, well below the wind terminal speed of $2250\,\mathrm{km\,s^{-1}}$. For X-ray emission which forms close to the star, the speed and X-ray flux are larger than can be easily reconciled with strictly self-excited line-deshadowing-instability models, suggesting a need for a fraction of the wind to be accelerated extremely rapidly right from the base. This is not so much a dynamical instability as a nonlinear response to changing boundary conditions. △ Less

Submitted 15 March, 2020; originally announced March 2020.

Comments: 18 pages, 4 figures; accepted for publication in the Astrophysical Journal

Journal ref: ApJ, 893, id52 (10p, 2020)

Closing gaps to our origins. The UV window into the Universe

Authors: Ana I. Gomez de Castro, Martin A. Barstow, Fréderic Baudin, Stefano Benetti, Jean Claude Bouret, Noah Brosch, Domitilla de Martino, Giulio del Zanna, Chris Evans, Miriam García, Boris Gaensicke, Carolina Kehrig, Jon La**ton, Alain Lecavelier des Etangs, Giampiero Naletto, Yael Nazé, Coralie Neiner, Jonathan Nichols, Marina Orio, Isabella Pagano, Gregor Rauw, Steven Shore, Gagik Tovmasian, Asif ud-Doula, Kevin France , et al. (1 additional authors not shown)

Abstract: The investigation of the emergence of life is a major endeavour of science. Astronomy is contributing to it in three fundamental manners: (1) by measuring the chemical enrichment of the Universe, (2) by investigating planet formation and searching for exoplanets with signatures of life and, (3) by determining the abundance of aminoacids and the chemical routes to aminoacid and protein growth in as… ▽ More The investigation of the emergence of life is a major endeavour of science. Astronomy is contributing to it in three fundamental manners: (1) by measuring the chemical enrichment of the Universe, (2) by investigating planet formation and searching for exoplanets with signatures of life and, (3) by determining the abundance of aminoacids and the chemical routes to aminoacid and protein growth in astronomical bodies. This proposal deals with the first two. In the Voyage to 2050, the world-wide scientific community is getting equipped with large facilities for the investigation of the emergence of life in the Universe (i.e. VLT, JWST, ELT, GMT, TMT, ALMA, FAST, VLA, ATHENA, SKA) including the ESA's CHEOPS, PLATO and ARIEL missions. This white paper is a community effort to call for the development of a large ultraviolet optical observatory to gather fundamental data for this investigation that will not be accessible through other ranges of the electromagnetic spectrum. A versatile space observatory with UV sensitivity a factor of 50-100 greater than existing facilities will revolutionize our understanding of the pathway to life in the Universe. △ Less

Submitted 2 November, 2019; originally announced November 2019.

Comments: White Paper submitted to the European Space Agency call for Voyage 2050. This WP proposes the development of a European Ultraviolet Visible Observatory (EUVO) to study the origins of life and it is largely based in arXiv 1306.3358 (ApSS,354,229)

A Swift view of X-ray and UV radiation in the planet-forming T-Tauri system PDS 70

Authors: Simon Joyce, John Pye, Jonathan Nichols, Kim Page, Richard Alexander, Manuel Gudel, Yanina Metodieva

Abstract: PDS 70 is a $\sim$5 Myr old star with a gas and dust disc in which several proto-planets have been discovered. We present the first UV detection of the system along with X-ray observations taken with the \textit{Neil Gehrels Swift Observatory} satellite. PDS 70 has an X-ray flux of 3.4$\times 10^{-13}$ erg cm$^{-2}$ s$^{-1}$ in the 0.3-10.0 keV range, and UV flux (U band) of 3.5$\times 10^{-13}$ e… ▽ More PDS 70 is a $\sim$5 Myr old star with a gas and dust disc in which several proto-planets have been discovered. We present the first UV detection of the system along with X-ray observations taken with the \textit{Neil Gehrels Swift Observatory} satellite. PDS 70 has an X-ray flux of 3.4$\times 10^{-13}$ erg cm$^{-2}$ s$^{-1}$ in the 0.3-10.0 keV range, and UV flux (U band) of 3.5$\times 10^{-13}$ erg cm$^{-2}$ s$^{-1}$ . At the distance of 113.4 pc determined from Gaia DR2 this gives luminosities of 5.2$\times 10^{29}$ erg s$^{-1}$ and 5.4$\times 10^{29}$ erg s$^{-1}$ respectively. The X-ray luminosity is consistent with coronal emission from a rapidly rotating star close to the log $\frac{L_{\mathrm{X}}}{L_{\mathrm{bol}}} \sim -3$ saturation limit. We find the UV luminosity is much lower than would be expected if the star were still accreting disc material and suggest that the observed UV emission is coronal in origin. △ Less

Submitted 1 November, 2019; originally announced November 2019.

Comments: 5 pages, 3 figures, accepted for publication in MNRAS Letters

Long-baseline horizontal radio-frequency transmission through polar ice

Authors: P. Allison, S. Archambault, J. J. Beatty, D. Z. Besson, C. C. Chen, C. H. Chen, P. Chen, A. Christenson, B. A. Clark, W. Clay, A. Connolly, L. Cremonesi, C. Deaconu, M. Duvernois, L. Friedman, R. Gaior, J. Hanson, K. Hanson, J. Haugen, K. D. Hoffman, E. Hong, S. Y. Hsu, L. Hu, J. J. Huang, A. M. -H. Huang , et al. (41 additional authors not shown)

Abstract: We report on analysis of englacial radio-frequency (RF) pulser data received over horizontal baselines of 1--5 km, based on broadcasts from two sets of transmitters deployed to depths of up to 1500 meters at the South Pole. First, we analyze data collected usingtwo RF bicone transmitters 1400 meters below the ice surface, and frozen into boreholes drilled for the IceCube experiment in 2011. Additi… ▽ More We report on analysis of englacial radio-frequency (RF) pulser data received over horizontal baselines of 1--5 km, based on broadcasts from two sets of transmitters deployed to depths of up to 1500 meters at the South Pole. First, we analyze data collected usingtwo RF bicone transmitters 1400 meters below the ice surface, and frozen into boreholes drilled for the IceCube experiment in 2011. Additionally, in Dec., 2018, a fat-dipole antenna, fed by one of three high-voltage (~1 kV), fast (~(1-5 ns)) signal generators was lowered into the 1700-m deep icehole drilled for the South Pole Ice Core Experiment (SPICE), approximately 3 km from the geographic South Pole. Signals from transmitters were recorded on the five englacial multi-receiver ARA stations, with receiver depths between 60--200 m. We confirm the long, >1 km RF electric field attenuation length, test our observed signal arrival timing distributions against models, and measure birefringent asymmetries at the 0.15% level. △ Less

Submitted 19 January, 2021; v1 submitted 28 August, 2019; originally announced August 2019.